jsd

@METR_Evals - Opinions my own.

Anonymous feedback:

[jsd @datagenproc]

Personal news: after two years at @EpochAIResearch, I’ve joined @METR_Evals to work on embedded assessments of AI R&D acceleration: going inside AI developers to measure how much AI is speeding up AI development, and publishing our findings.

[jsd @datagenproc]

@peligrietzer Thanks!

[jsd @datagenproc]

How it started / How it’s going

[jsd @datagenproc]

@mattyglesias @lxeagle17 Maybe @NateSilver538 @ZacharyDonnini @PoliticalKiwi will have takes

[jsd @datagenproc]

@mattyglesias @lxeagle17 Something like this?

[jsd @datagenproc]

@tobyordoxford I’m at METR but I don’t have private info.

[jsd @datagenproc]

Note also that the authors confidently exclude training compute acceleration as a cause of the claimed AECI acceleration.

The evidence that led them to that conclusion is unfortunately redacted, and it’s fair for people to differ in how much they want to defer.

But I think that’s relevant context with respect to your alternative explanation.

[jsd @datagenproc]

@testingham Some have not yet given up, and are still fighting the good fight (jk)

[jsd @datagenproc]

@fleetingbits [I don’t have private information]

Re info on Ant internal agent usage, doesn’t the S1 give a doubling time of 30 days? 250x from January to September. Page 188 of the S1/A1.

[METR @METR_Evals]

We reviewed the “Risks from automated R&D” section of Anthropic’s August 2026 Risk Report, with access to the nonpublic version.

We focus on the report’s claims about catastrophic risks from automated AI R&D and about whether Anthropic had crossed relevant capability thresholds.

[METR @METR_Evals]

Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development.

We are now publishing a report summarizing the results of this risk assessment.

[jsd @datagenproc]

@usmananwar391 @MaxNadeau_ @JakeMendel99 So, conceptual model

[jsd @datagenproc]

@usmananwar391 @MaxNadeau_ @JakeMendel99 I think Max is referring to The Behavioral Selection Model https://t.co/Vxw6nm3FRx

[jsd @datagenproc]

@johnhewtt "An Interpretability Illusion For BERT" from Bolukbasi et al. 2021 is a relevant citation here

[jsd @datagenproc]

@chi_t_williams Incredible

[jsd @datagenproc]

@Jess_Riedel @davidshor @stevenmklein Yeah, that seems right to me

[jsd @datagenproc]

@davidshor @MattZeitlin TIL!

[jsd @datagenproc]

I think Confidence Intervals are too often a ritual where the person adding them doesn’t actually know what question they’re answering, it’s just a thing you do.