Keith Dear

“Clever but dangerous…no real combat experience”

[Keith Dear @kpd_musing]

This is huge— @Cassi_on_X came 2nd overall and 1st on "Dataset" questions on @Research_FRI Forecastbench! 🥈📈

Only @elonmusk's @xai @grok outperformed us overall.

What this means: 🧵

[Ed Leon Klinger @edleonklinger]

What's about to happen in artificial intelligence is so sudden and extreme and unusual that it's almost impossible to describe it without coming across as slightly unhinged. Very soon, perhaps within a matter of months, AI may well be building its own upgrades at such a rate that it will get away from us humans entirely.

On the plus side this would mean an astronomical acceleration in mathematical breakthroughs, drug discovery, manufacturing, perhaps even new areas of science that we've never explored or understood. We will be ushering in a golden age of abundance and splendour that even the greatest sci-fi authors could never have imagined. Robots, flying cars, telepathy, immortality, digital consciousness - it’s actually going to happen, and it will be incredible!

On the downside, virtually every job that can be done on a computer will become automated overnight. Waves of redundancies, entire industries and professions made obsolete, rapidly widening economic inequality and unparalleled wealth concentration, accompanied by discontent, protests, looting - that may well happen too.

These massive cultural and economic shifts will occur concurrently with a wave of cyberattacks the likes of which we have never seen. Energy grids, airports, biolabs, supermarkets, transport infrastructure, these are no match for the next generation of superintelligent agent swarms that go rogue, or are leveraged against us by our political adversaries. Expect blackouts, viral outbreaks, lockdowns, entire cities and countries going dark.

And all of this will happen in an insanely short period of time, an absolutely dizzying rate of cultural, economic, and physical change that will leave the average person feeling disoriented and confused, like 100 COVID pandemics all happening at once.

I say this not because I am a “doomer” or because I want to encourage panic, but simply because it feels increasingly likely, and yet drastically underreported save for a small AI-obsessed enclave here on X. The disconnect between what’s about to happen, and the average person’s understanding of what’s about to happen, has never been wider.

[Kanishka Narayan MP @KanishkaNarayan]

AI is powering a new era of British growth. And this is just the beginning.

I'm determined to seize this opportunity to reindustrialise Britain, create good jobs, free up money for public services, and spread prosperity to every corner of our country.

[Keith Dear @kpd_musing]

@madvixen83 @TimesRadio Thank you! It will be - look forward to picking up next week on the show!

[Uriah @crimkadid]

Historians used to bicker about the origins of capitalism, a debate that confuses modern readers who assume "capitalism" is a universal. But the old-timers were more right than wrong: there were once $20 bills lying on every street and, mysteriously, no one picked them up.

[Andrew Curran @AndrewCurran_]

They now predict that 95% of Anthropic's R&D will be fully automated by November 2027. This is what Anthropic describes as AL5, fully autonomous RSI. This would have many implications for policy, as this will eventually happen at every lab, not just Anthropic

[Ethan Mollick @emollick]

Looks like Fable 5.2 is above the already steep trendline on three of the most informative unsaturated benchmarks (ECI is a composite of other benchmarks, Remote Labor measures the ability to do end-to-end freelance jobs, RoboDojo involves controlling robotics).

[Mehdi (e/λ) @BetterCallMedhi]

dans un silence assourdissant claude vient de lâcher une dinguerie en physique théorique en réglant 2 problèmes qui saoulaient les chercheurs depuis des décennies avec le coefficient de polyakov, il a capté que les trous noirs peuvent aussi pop par transition ultra rapide dans des tout petits univers et à côté de ça il a posé la limite exacte du fameux modular bootstrapping (je vous laisse chercher) pour faire le grand ménage dans toutes ces théories de gravité quantique un peu bancales qui tournaient en rond

pour vous la faire simple, imaginez que les physiciens galéraient à assembler des pièces de lego cosmiques super complexes sur la façon dont un mini trou noir se forme dans l’univers, bah dites vous queclaude a juste sorti la formule miracle pour tout emboîter d’un coup et jeter toutes les fausses pistes à la poubelle et je trouve ça juste incroyable à observer putain!!!

ce qui rend ce truc complètement fou c’est que c’est le genre d’avancée qui va pour moi irriguer toute la tech de pointe pour les années à venir et ceux qui croient encore que l’ia sert juste à gratter des mails ou à faire des résumés de réunion sont totalement perchés

comprenez qu’on est en train de basculer dans une époque où l’IA devient le moteur absolu de la découverte scientifique

[AI Notkilleveryoneism Memes ⏸️ @AISafetyMemes]

"This trajectory, if continued, means a world-changing intelligence explosion in 2027, so the public needs to know."

Every chart right now is a screaming red alert.

[Pushmeet Kohli @pushmeet]

Announcing a major milestone achieved by my team @GoogleDeepMind: we have proven key security properties of the machine code underlying the Signal messaging protocol. This is the first time the actual production code of a widely deployed protocol has been formally verified – by any method, with or without AI – and at this depth.

More than a research breakthrough, it’s a prime example of how frontier AI can be a force multiplier to counter the speed and scale of modern cyber threats.

Formal verification offers a higher standard of security; however, it’s extremely complex and time consuming, which is why in many cases it’s considered to be impractical. Similar efforts could take teams decades of combined effort. We did it in a month’s time.

Using AlphaProof Code, Google DeepMind's new agentic framework for formal verification that runs on LLMs including Gemini Carbon, we verified the Signal protocol down to the actual code running on devices.  This milestone lays the groundwork for a future in which vital systems, from medical device firmware to power grid software, ship with mathematical proof that their protections work.

Reaching that future will take continued research from our community. In that spirit, we're open-sourcing our formal specifications and verification artifacts so others can build on this work.

On a personal note, this moment is of deep significance to me. I started my own research journey more than two decades ago as an intern working on checking the consistency of complex software systems. Seeing this grand challenge made practical with modern AI is extraordinary.

I'm extremely proud of my team members: Swarat Chaudhuri, Andrew Ferraiuolo, Anton Kovsharov, Tej Chajed, Sam Schlesinger,  Karn Seth, Jonathan Katz, and Charles Sutton for taking on this grand challenge and delivering, and grateful to the Signal team for their partnership.

https://t.co/kN1eRYQ8fF

[Archie Hall @ArchieHall]

AI will be the first tech capex boom that Britain misses out on in the modern era

[sam @discoplomacy]

MIDDLE POWERS - Australia especially - this is your time to get in on the compute rollout. Get those frontier companies diversifying into your country ASAP.

UK too but more bearish on stuff getting built.

[sam @discoplomacy]

Over the last year and a half I’ve tried to string together the following articulation for Britain’s general AI strategy and posture. It is something like this:

> The British Government is developing a new form of technological statecraft.

> Unable to compete directly with the Americans + Chinese in building frontier AI, it has begun strategically establishing specialist institutions that give it insight into and occasional influence over the technology's development and application.

>The first part involves setting up government-backed organisations that can help Britain understand the technology, and potentially create ways of testing it that lead to safer and more secure developments. This is what AISI does - the world’s leading Govt-backed org in this space.

> This is part of a strategy to buy a seat at the US-China AI table too.

> The second part is now beginning. It involves taking the information and analysis, and trying to apply it at new AI intersections with Govt x private sector crossover.

> Take our new AI Biology Lab. It has been founded with two points in mind: help protect Britain from future pandemics (including potentially AI-created or enhanced ones), and help develop a biology industry at the frontier of this technology, tapping into deep existing strength across Government and the private sector. I expect it to be successful.

> A point that is often missed: democratic countries will need to be able to evidence that they have taken meaningful action in the event of an AI-related incident that cuts across into their voting population’s radar. UK is arguably pioneering a model to try and also get ahead of this too.

> Yes, it is not a perfect answer. More AI-compute, better salaries, less regulation on building, more tapping into cyber stuff etc etc, all points raised by others and myself before would be good. I think UK’s rollout on this is v short of where I would like it to be.

> But this is the situation we find ourselves in. Much like foreign policy, governance behind the frontier is a choice between bad and very bad very often. A strong strategy involves trying to tip the odds more in your favour whenever possible.

[Ben Brittain @BenABrittain]

Agree. Britain has the rare stuff other countries can only dream of: talent, institutions, capital, global reach. What it lacks is the basics: over regulated, over taxed, grid, power, housing, transport, the ability to build quickly. That part is fixable.

[Julia Willemyns @jujulemons]

I often have policymakers tell me that they don't think they need to worry about AI infrastructure or supply chains because the real gains from AI will come from adoption.

However highly you rate adoption, there is something quite striking about the fact that the countries where the AI policy debate is most defined by an 'adoption debate' are often the ones furthest behind on adoption.

In part, this is because AI adoption tracks economic competitiveness more widely, and Europe has struggled with competitiveness for a long time. An 'adoption play' may just look like good economic policy: loosen up labour markets, reform regulation, increase business dynamism.

The Americans are debating building data centres, securing supply chains and managing cyber risks, not adoption. They aren't relying on elaborate government strategies to encourage firms to use AI. And yet American firms are adopting AI!

[Robocurve PBC @robocurve]

We asked two isolated copies of the same AI to exchange a secret word with only a robot and a camera.

They succeeded by using air-writing, Morse code, flag semaphore and a 5-bit binary alphabet, without agreeing on a protocol beforehand.

[Illimitable Man (IM) @SovereignIM]

A civilization in which the poor live more comfortably but have almost no realistic prospect of becoming powerful, independent or exceptionally successful through their own merits is not a superior civilization - it is the death of meritocracy in service of mediocrity under the banner of efficiency. Why would anyone bother trying to become the best at anything? There is no point.

A machine can do it better so it's not economically viable to do anything other than own the compute that powers that machine, meaning even if you're really good at it, it won't do anything for you - class positions becomes entrenched, with no amount of effort or talent being enough to transcend them if you're on the wrong side of that line, because the market will stop rewarding your talents.

Millions upon millions of people who define themselves by their jobs will be left purposeless. Most of humanity will be a serf class collecting welfare with no incentive to be brilliant or excellent or bother with anything, and will drive themselves mad descending into infinite leisure.

I want to be wrong about this. But I don't think I am. After all the condescension of the flawed analogy between the car replacing the horse, and then the cope about adapting by keeping a human in the loop and augmenting your workflow, only to end up at "so yeah you were right actually you will be made obsolete" is just fucking wild.

[Prof. Frank McDonough @FXMC1957]

31 October 1973. 1st episode of the 26-part classic ITV documentary series World at War was broadcast. It was narrated by Laurence Olivier with music by Carl Davies. It cost £900,000 (worth £11 million today).

[Richard Sutton @RichardSSutton]

One reason I am sanguine about the technological singularity is that I have seen it coming for over 50 years. In 1975, when I was 17, I wrote:

"Before man can most effectively aid his problem solving machinery, he must first understand it. At present, his understanding is very poor. However, when the breakthrough in understanding the brain occurs, improvement will come very fast and in an exponential increase. Once man has created an intelligence greater than his own, that intelligence should be able to create an intelligence greater than itself, as man whose intelligence is less than its was able to create its intelligence. Thus, the level of intelligence will shoot up with incredible rapidity, probably within the space of years. Within a few years of a breakthrough, which may occur any time now, man will either have been left behind by a superior intelligence of his own making, which he can no longer understand, or he will have become that intelligence and will no longer be like anything imaginable today."

[Under Secretary of State Jacob S. Helberg @UnderSecE]

The decisions we make on Super Intelligence today will shape future generations. History shows that during technological paradigm shifts, early adopters surge ahead. Singapore is moving early. Singapore joined Pax Silica, the trusted coalition building our tech future, and signed new MOUs with U.S. companies to bring Super Intelligence to solve real business problems across Southeast Asia.

We welcome Singapore’s close work with American firms to deploy SI for public services, develop security standards for agentic models, and create an application-layer ecosystem that builds on U.S. frontier models.

[Joshua Achiam @jachiam0]

People who have been in research fields develop a felt sense that most "research" (in almost any field that isn't exactingly technical) is research theater, and many superstar careers enjoy star status not because of a unique connection to the truth but because of a unique connection with an audience. No one really has the time or stature to, from a position of objectivity and trust, bring popular researchers to heel. But it will get increasingly hard to hide from serious critique. The machines can do critique now, at the level of depth and rigor previously only possible for the academically deranged. We're going to have to revise a lot of what we "know" about the world that isn't really knowledge but is actually just received popular consensus - like floating voltages without grounding wires. The grounding wires are coming.