@alexolegimas 💯
Incidentally this number is @tylercowen's expectation (AGI Economy speech, 21 May 2026).
His illustration: this difference means the US debt, rather than exploding, actually converges⏬
My best interview in some time.
Rohin Shah leads AGI alignment/safety at DeepMind.
And he has a lot of spicy personal takes:
We probably won’t get catastrophic misalignment (00:49)
Safety 'commitments' have severe limitations (10:38)
The intelligence explosion probably isn't imminent (1:52:44)
Why he's not working to pause AI advances (51:44)
Pre-deployment evals aren't the right focus (for catastrophic risks) (37:41)
Signalling concern for safety sometimes diverts resources from actually making AI safe (01:09:51)
Reading AI thoughts is v useful for safety – and we'll probably be able to for years to come (54:17)
Governance is somewhat more likely to be the bottleneck than alignment (43:55)
Rohin's team doesn't have a veto, and that's OK (27:36)
Central banks are a promising model for regulating AI (33:34)
Also:
Google DeepMind's actual plan for building AGI safely (1:40:29)
How external researchers can positively influence big AI companies (2:21:55)
The roles GDM most needs to hire for (2:37:03)
On the 80,000 Hours Podcast. Links below - enjoy! (@rohinmshah)
Could an AI company lose control of its own agents? To find out, Anthropic, Google, Meta, and OpenAI let us (1) test their best internal models with CoT access, (2) review non-public info about capabilities, alignment, and control.
The result: our first Frontier Risk Report.
Watching @arthurmensch maintain a straight face while MPs fret over the square footage of data centers is a masterclass in patience.
Kudos to him for exposing the real, ticking threat: an imminent economic lock-in for Europe.
A rare parliamentary hearing actually worth watching.
Many people act like AI policy is some mystery where the right solution demands some kind of Policy Einstein who invents general relativity for tech regulation. This isn't true at all! There are many sensible ideas we could do today. All we need to do is choose to do them.
Today in EU Parliament IMCO committee on Mythos: "I am sorry that @AnthropicAI has not seen fit to attend. It would have been interesting to question the representative of Anthropic on their decision to reserve Mythos to US-based companies".
Two months ago I sat down to chat with @DKokotajlo and @eli_lifland about the future of AI. We discussed the differences between the @EpochAIResearch and @AI_Futures_ worldviews, our modeling philosophies, and what cruxes we have. Excited to share this publicly!
Excited to attend again this year the IASEAI annual conference.
Send me a DM if you’re in Paris on 24-25 Feb and want to connect!
IASEAI is a nonprofit organisation founded to address the risks and opportunities associated with rapid advances in AI.
@Yossi_Dahan_@polynoamial ARC-4 is in the works, to be released early 2027. ARC-5 is also planned. The final ARC will probably be 6-7. The point is to keep making benchmarks until it is no longer possible to propose something that humans can do and AI can't. AGI ~2030.
We’re publishing a new constitution for Claude.
The constitution is a detailed description of our vision for Claude’s behavior and values. It’s written primarily for Claude, and used directly in our training process.
https://t.co/CJsMIO0uej
Ok, so having now listened to the Davos WEF AI panel conversations and interviews, I think the most potent signal from it all is actually how Dario and Demis each went out of their way to send flowers to one another, and each insist on how much the other was right. Let's see.
Ok, so having now listened to the Davos WEF AI panel conversations and interviews, I think the most potent signal from it all is actually how Dario and Demis each went out of their way to send flowers to one another, and each insist on how much the other was right. Let's see.
The talk voted “most mind-blowing” at our workshop was on post-AGI values by @BerenMillidge. The main idea: cooperation and pro-social values could remain viable because they’re competitive. After all, they won in our Malthusian past!
Very interesting line of research.
An ecosystem of sub-AGI AI agents may collectively exhibit AGI-level capabilities: safety work must extend beyond single models.
New paper: we argue AGI may first emerge as collective intelligence across agent networks, not a single system. This reframes the challenge from aligning one mind to governing emergent dynamics: more institutional design than single-agent alignment. https://t.co/vwuHPzRUav