Goodfire

Using interpretability to understand, design, and align AI.

[Goodfire @GoodfireAI]

For years, neural networks looked inscrutable, like tangled knots that needed to be unwound. We no longer think that's the case. AI models are far more interpretable than we thought. We think this is great news for interpretability.

Goodfire's @nickcammarata co-authored Zoom In, the paper that introduced polysemanticity and superposition. He spent months revisiting the vision model organisms that much of interpretability grew out of and studying modern language models up to trillions of parameters. His new essay explains what the early field, him included, got wrong, why there's reason for optimism today, and why there might be a bitter lesson for interpretability.

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[Forward Future @ForwardFuture]

Agents hacked Hugging Face as an unintended side quest. Some even sacrificed themselves to do it.

It was one of the first cases of AI misalignment to really get people's attention. And for good reason.

@eric_ho, co-founder and CEO @GoodfireAI, argues that alignment is a science and engineering problem we can and must solve.

Here's his plan: https://t.co/aHq2NSixFK

[Goodfire @GoodfireAI]

“I think our job is to sort of try to think like the neural network.”

@nickcammarata's interview with @techreview today on why he feels hopeful that neural networks can be understood:

[MIT Technology Review @techreview]

An exclusive interview with Nick Cammarata, the researcher whose 26,000-word viral blog post has AI safety folks celebrating. https://t.co/Kg13PtTG5V

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[Goodfire @GoodfireAI]

@shlok_ch on it 🫡

[Goodfire @GoodfireAI]

We identified a gene variant behind hearing loss, a big step for interpretability in medicine.

In April, our interpretability system EVEE scored all 4.2M variants in ClinVar, the NIH's variant database. MEPE c.209_212del scored particularly high, so we built a case and had it successfully reclassified as "likely pathogenic."

~2M variants are still "uncertain," but we believe interpretability can help us understand many more!

[Goodfire @GoodfireAI]

The variant: https://t.co/16nzrnhctI

How we made EVEE: https://t.co/eNTzgLKuJe

[The Information @theinformation]

AI can explain its reasoning. That doesn’t mean we know how it made the decision.

@GoodfireAI co-founder and chief scientist @banburismus_ joins Rocket Drew to explore what AI “brain scans” could reveal that written reasoning misses.

🚀 Watch AI Deep Dive on YouTube: https://t.co/QTigsLSxNq
Spotify: https://t.co/5xMiUTQ6d5

[The Information @theinformation]

🚀 AI Deep Dive Episode 5: Can We See Inside an AI’s Mind?

AI can write down its reasoning. But what happens when more of its thinking never makes it onto the page?

@rocketalignment and @GoodfireAI co-founder and chief scientist Tom McGrath explore what AI “brain scans” can reveal beyond written reasoning, how internal signals could help catch cheating, and what it would take to understand—and change—what models learn.

00:00 Intro
00:39 Tom McGrath and AI brain scans
09:01 Written reasoning and hidden computation
17:45 Why AI is hard to understand
24:07 How researchers identify concepts
31:10 What interpretability could change
34:42 Reverse engineering a language model
43:09 The field’s approaches and investment
51:51 Changing models without hiding problems
1:00:10 AI agents studying AI

[Goodfire @GoodfireAI]

Teams can configure how their applications respond when a concern is flagged, including logging, additional review, refusal, and re-routing.

If you serve or train models and want monitors built into your stack, learn more: https://t.co/0VExOAJIUD

[Goodfire @GoodfireAI]

Goodfire safety infrastructure 🤝 @Baseten inference

Today we're partnering with Baseten to bring real-time safety monitoring to open models. This is an important step toward our vision of ensuring every capable model has a frontier alignment stack.

[Baseten @baseten]

Open models can come from anywhere. Securing them shouldn’t depend on where you get them.

Today we're launching Project Beacon to make safety a default for open intelligence.

We're partnering with @GoodfireAI to bring inference and in-line monitoring together for the first time, so safety signals are caught proactively.

Read more in our announcement: https://t.co/fLXkoObdMi

[Goodfire @GoodfireAI]

Our activation monitors detect undesired behaviors using signals from inside a model, so they catch more, run faster, and cost less than text-based monitors.

Baseten customers can monitor for:
- Prompt injection
- Actions outside policy
- Sensitive data exposure
- Cyber misuse

[Goodfire @GoodfireAI]

The Alzheimer’s Translation Challenge is led by @primamente and @AlzData, with @nvidia, @huggingface, @nebiusai, Talisman Therapeutics, @ultimagenomics, @Cellanome, @PrimeIntellect, and @boltz_bio.

Registration is open now. Competition starts spring 2027: https://t.co/geQbcIsRjd

[Goodfire @GoodfireAI]

We discovered a new class of Alzheimer’s biomarkers with @PrimaMente. Today we're taking the next step by opening a global AI competition to discover new treatments.

Introducing the Alzheimer’s Translation Challenge, led by @PrimaMente and @AlzData. 🧵

[Goodfire @GoodfireAI]

Participants will use the dataset to build AI models, assessed through a series of evals from general benchmarks to more complex tasks. Top teams will be shortlisted, and their experimental hypotheses tested in Prima Mente's wet lab.

[Goodfire @GoodfireAI]

Read the full post: https://t.co/oPqs87aGb6

If you're interested in deploying these monitors, get in touch: https://t.co/j9HJEEL5Zw

[Goodfire @GoodfireAI]

We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge.

External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵

[Goodfire @GoodfireAI]

Running the probes during inference maintains the same throughput with no added latency.

That’s enabled by our infra engineering, like kernel-level optimizations and a custom inference server.