Dane Sherburn

Co-founder @pzeroresearch. Previously on Preparedness team @OpenAI (SWE-bench Verified, MLE-bench, PaperBench) and a scholar @MATSprogram.

[Dane Sherburn @danesherbs]

Read more: https://t.co/VOmUIOpyYr

[Dane Sherburn @danesherbs]

Super excited to announce TasteVal, which measures the experimental research taste of frontier models!

https://t.co/OBMYUgwrj5

[P-Zero Research @pzeroresearch]

How fast is AI's research taste improving?

We find that the experimental research taste of frontier models has doubled every ~3 months since December 2025. The best model, Opus 5.5, now exceeds our expert human baseline. Our human experts are experienced researchers, but most haven’t worked at a frontier lab.

Why measure research taste? In the AI Futures Model, it largely determines how quickly artificial superintelligence is reached once coding is fully automated.

[Dane Sherburn @danesherbs]

This is part of a broader effort to measure the research taste of frontier models, which, according to the AI Futures Model, is the most sensitive parameter determining when artificial superintelligence is developed once coding is automated.

[Jack Clark @jackclarkSF]

"stochastic parrot" was a mimetically-fit cognitive virus that spread from 2021-2025; it temporarily blinded many gifted people to the nature of AI progress, burning up crucial years in which they could have helped think through the response to the situation.

[Dane Sherburn @danesherbs]

@jasminekd01 You're doing such incredible work Jasmine! 💪

[bilal 🔶 @bilalchughtai_]

I recently resigned from Google DeepMind, where I worked on AGI safety and alignment research. At Google, I witnessed AI development first hand. I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.

The pace of AI progress in the past few years has been staggering. When I first started working on AI in early 2022, AIs were amusingly useless. Just four years on, AI agent swarms from OpenAI are cracking famous century-old math problems and, more worryingly, escaping the control of OpenAI and autonomously hacking into the third-party company HuggingFace, against anyone's wishes.

Things will only get crazier: I think it's possible that the AI companies might, in the next few years, succeed in building superintelligent AI systems that far exceed human capabilities in every domain. I am not confident that these AI systems will do what we want. In particular, misaligned superintelligences may, much like the rogue AI agents involved in the HuggingFace incident, escape our control and take dangerous actions that may result in the permanent disempowerment or death of humanity. Alignment is the problem of preventing this, and is both difficult and unsolved. Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment.

I am optimistic that navigating AI safely is possible. In order to do so, we need to coordinate to avoid this manic race between AI companies. We need to pace AI development to a speed that society can handle, where emerging risks can be addressed before extreme harm is realised. We need much more transparency into AI development to ensure that AI companies are not imposing unacceptable levels of risk on us all.

More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense. I'm very directly working on this next: I want to help people interested in working on mitigating catastrophic AI threats do the most effective work that they can. I think many people from many backgrounds in many roles have a part to play.

[Jérémy Scheurer @jeremy_scheurer]

Join us as founding Staff in the DC Office!

[Dane Sherburn @danesherbs]

@MaskedTorah @NathanpmYoung Curious what properties you care most about in a third-party evaluator?

[Daniel Kokotajlo @DKokotajlo]

Wow. Probability of Plan D just went down again, probability of Plans C, A, and S just went up!

[Dane Sherburn @danesherbs]

@sjgadler Congrats Steven, so happy for you! Enjoy :)

[Alex Turner @Turn_Trout]

@JohnCornyn Senator, AI 2040 presents a credible starting point and works even if China tries to cheat on a treaty. The plan allows Americans to benefit from innovation & stay ahead but protects against the worst harms.

https://t.co/JtCwqX2oDj

[Drake Thomas @MaskedTorah]

@jillgun I would burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive. I expect a great many of my colleagues across the industry would as well.

I promise you, we are actually just fucking scared, it's not galaxy brained marketing.

[catherine is in sf ʕ•ᴥ•ʔ-☆ @wilhelmscreamin]

https://t.co/D6Kwj5e7FR

[Benjamin Todd @ben_j_todd]

If leading technical experts on AI say alignment is not on track and it could kill everyone, the first reaction should be "oh shit!!!", not "your org is bad at comms".

[Evan Hubinger @EvanHub]

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

[Dave Banerjee @DaveRBanerjee]

Applications for the IAPS AI Policy Fellowship are now open!

I did the fellowship last year and it was a lovely experience

And if you're accepted, you can potentially work with me on preventing extreme power concentration! :D

[Dane Sherburn @danesherbs]

@OwainEvans_UK @BetleyJan @jameschua_sg @ajeya_cotra @MaxNadeau_ @coeff_giving Well deserved, Owain! Congrats! :)

[Nikola Jurkovic @nikolaj2030]

There is no good reason to expect that we will be able to align or control the first superintelligences. AGI companies are rushing to create superintelligent AI. The default plan is that humanity is destroyed, likely sometime around 2030.

I don't know how we can reliably survive this decade. I think that stopping the race to ASI and doing something like Plan A or Plan S should plausibly be humanity's top priority.

[Benjamin Todd @ben_j_todd]

It's almost as if the world is not taking AGI seriously enough