Charles Foster

On policy @METR_Evals 🧪 “excels at reasoning & tool use” 🪄 CoI disclosures on my Substack “About” page.

[Charles Foster @CFGeek]

I often joke to people that I tweeted my way from “random AI researcher who likes open models” to a job at METR. But I don’t think I’ve told the story to y’all before. Time to do that: 🧵

[Charles Foster @CFGeek]

@BGOV To save you a click, here's the public disclosure: https://t.co/5hezBocJuB

As the disclosure mentions, this engagement is to help METR meet policymakers and explain what the org does. METR doesn't advocate *for* anything, nor is Valcour advocating for anything on METR's behalf.

[EvalEval Coalition @evaluatingevals]

🚀 Join Evaluating Evaluations this Saturday, Oct 24, for Embedding Evaluators in Practice, a flash micro conference on embedded AI evaluators! 🎤 Come in person at UCSF Mission Bay 🌉, join our London watch party 🇬🇧, or tune in online 💻 🧵👇

[Charles Foster @CFGeek]

@harryboothtime To be clear, I think an external assessment of *information security* at a frontier AI company would look pretty different. I would be excited for cybersecurity firms to get into that game, but unfortunately haven’t yet seen evidence that they’re trying/doing this.

[Charles Foster @CFGeek]

@DavidSacks Agreed

[Charles Foster @CFGeek]

On July 21, a two outside researchers started their first day embedded at OpenAI to examine at how it monitored coding agents.

The very same day, OpenAI disclosed the HF incident, which it said “points to the need to further strengthen […] monitoring during internal testing”.

[METR @METR_Evals]

We analyzed monitoring as it existed at the start of the pilot, coincidentally the same day OpenAI disclosed the Hugging Face incident. We didn't evaluate later changes, like expanding monitoring to RL training, which wasn’t monitored at the pilot's start.
https://t.co/oTtU8TIyoo

[METR @METR_Evals]

Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.

[Charles Foster @CFGeek]

@DeltaTeePrime @_NathanCalvin Not always. METR had this right in the Frontier Risk Report pilot and OpenAI / Hugging Face report, but other work was done under normal NDAs without a specific carveout for this.

[Charles Foster @CFGeek]

Seems good!

[NSA Cyber @NSACyber]

NSA joined @NIST and @CISACyber in releasing guidelines for managing SI control incidents. Organizations utilizing SI are encouraged to adopt these practices. Read the joint report: https://t.co/9Bd3Sku0KW

[Charles Foster @CFGeek]

@BronsonSchoen Good idea! Tim tried this out https://t.co/b7p2eD4SkN

[Tim Hua 🇺🇦 @Tim_Hua_]

We believe that Anthropic should include "undermining of safety-relevant decision-making" as key risk pathway.

This is partially inspired by Section 2.20 of their risk report, where Anthropic asked Claude for feedback on their report, and received a (imo) sycophantic answer.

[Charles Foster @CFGeek]

@dfrsrchtwts Lmao

[Charles Foster @CFGeek]

My colleagues just published a 69-page response to Anthropic’s report on alignment.

They agree about the risk level but “are not confident that, if risk were higher, the methodology in [Anthropic’s] report would reliably identify this.”

List of disagreements starts on page 25.

[METR @METR_Evals]

We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis.

[Charles Foster @CFGeek]

@_NathanCalvin (1) I’d hope! But by default, NDAs give no guarantee that it’d be disclosed when a publication isn’t approved. That’s a reason why it’s important to disclose approval rights.

(2) In the pilot Frontier Risk Report, companies had the right to exit silently

https://t.co/9BmGx7QJuF

[Charles Foster @CFGeek]

Companies could exit from the pilot up to the point where they signed off on (redacted and/or anonymized) evidence, but not after. That meant companies knew exactly what non-public evidence we might cite, but couldn't directly control our downstream conclusions, framing, or tone.

[METR @METR_Evals]

We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis.

[Charles Foster @CFGeek]

@eventidia One day! ™️

[Charles Foster @CFGeek]

@HamelHusain Whoa!

[Charles Foster @CFGeek]

@TencentHunyuan Thanks for the disclosure! Which third-party models were involved in the activity?

[Charles Foster @CFGeek]

@jlkoepke Mm yeah I don’t care about creating an industry per se. Top priority is meaningful assurance

[Charles Foster @CFGeek]

@jlkoepke Is that a good thing?

[Charles Foster @CFGeek]

@Sauers_ Very cool, though I feel like it’s massively burying the lede to not say the bottom line up front: this is because these probes (accidentally) change the activations, and you can teach a model to detect these changes.