Jonathan Gabor

MTS @METR_Evals, opinions are my own

[Chris Painter @ChrisPainterYup]

I’m building a team in DC. Everyone in Washington has a stake in ensuring that reliable info about alignment and RSI reaches the public.

We need trustworthy communicators who can explain METR's loss-of-control risk assessments and goals to policymakers.

[METR @METR_Evals]

Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.

[roon @tszzl]

I almost missed this! big development!

[Elon Musk @elonmusk]

@AnthropicAI @METR_Evals SpaceX will probably do the same

[Jonathan Gabor @jpgabor]

@franudar @ArjunK100 @AnthropicAI @METR_Evals Assuming funding is your biggest concern, the vast majority of our funding has not come downstream of CG in any way. And the links people try to draw between CG and Anthropic seem tenuous at best.

https://t.co/84tiSj3K8u

[Jonathan Gabor @jpgabor]

@ArjunK100 @AnthropicAI @METR_Evals I'm not aware of anyone currently at METR who used to work at Anthropic?

Joe Benton will be joining us soon, he'll be the first one.

[Elizabeth Barnes @BethMayBarnes]

A common misconception about METR’s work with companies is expecting the relationship to be like working with a law firm or cybersecurity firm who are being paid to serve the interests of the company - METR is a non-profit and serves the public. For example, you can have your law firm refrain from working with your competitors. However, the standard practice in other evaluation regimes (e.g. bank examiners, financial auditors, NRC inspectors) is that assessors will move between multiple companies, and there are rules about how long an individual can stay with one company to prevent them “going native” and losing their independence or fresh perspective.

[Jonathan Gabor @jpgabor]

"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."
--Richard Feynman, appendix to the Challenger disaster report, 1986.

Couldn't be prouder to work at @METR_Evals.

https://t.co/tQtkpT7PDa

[METR @METR_Evals]

If an AI developer was on track to lose control of its internal agents, would anyone outside the company know?

Today, we are sharing METR’s vision for how we would work with AI developers to openly assess and share evidence about this.

[Elizabeth Barnes @BethMayBarnes]

METR’s goal: work w frontier developers to get evidence to public on unreleased models’ loss-of-control risks. Asks: 1.transparency on scope, redactions + limitations 2.staff can freely share info w us on in-scope topics 3.rapid publication. They control redactions + access given

[METR @METR_Evals]

If an AI developer was on track to lose control of its internal agents, would anyone outside the company know?

Today, we are sharing METR’s vision for how we would work with AI developers to openly assess and share evidence about this.

[Vals AI @ValsAI]

Xiaomi open-sourced the RL environments behind MiMo v2.6. In two-thirds of the coding tasks, the answer is still in the task's Git history, and MiMo finds it.

[david rein @idavidrein]

It seems to me like the Harbor framework for agent evaluations (which is used for Terminal Bench) allows agents to arbitrarily modify their trajectories before they are evaluated. Am I missing something?

https://t.co/jv4JVuEtyp

[Chris Painter @ChrisPainterYup]

It was an honor to speak today before Chairman @HawleyMO, Ranking Member @AndyKimNJ, and the subcommittee on METR's work, recent AI agent incidents, and the importance of transparency in frontier AI development.

[MTS @MTSlive]

METR AI researcher @ReillyHaskins02 reveals how METR can now freeze an AI agent mid-eval when its next action looks dangerous:

"As we've seen recently, eval inference can be prone to causing real-world harm sometimes. And so we wanted to try and reduce the likelihood of this happening during our evals."

"We made a per-action blocking monitor that runs live during select evals. It takes every action before it gets executed by an agent, and it scores it on some threshold of suspicion for causing real-world harm. If it's above some threshold, then it gets blocked for human review."

"The aim is to stop actions which would cause harm from actually being executed without being reviewed by a human first."
@METR_Evals

[Jonathan Gabor @jpgabor]

@S1r1u5_ @JeffLadish @PalisadeAI (this is one of the guys who got access to OAI’s codebase)

[Charles Foster @CFGeek]

In case you missed it, these are the folks who fund us at METR! You can find the same info and a bit more on our website.

IMO my colleagues have been quite thoughtful about whose funding we have (and haven’t) taken.

[Kelsey Piper @KelseyTuoc]

a lot of people work at METR so you can do the left version of this where they're all white men too if you want:

[Nate Silver @NateSilver538]

There are a lot of criticisms you can make of EAs (indeed, I make some in my book) but I don't think the EAs are particularly woke and the rationalists more broadly tend toward being anti-woke. Like they're not doing land acknowledgments at the Manifest conference.