Eric Ho

co-founder / ceo @GoodfireAI - AI interpretability research company

[Eric Ho @eric_ho]

Technical alignment is a science and engineering problem that we can and must solve, and interpretability is the bottleneck. I wrote up Goodfire’s plan to get there. https://t.co/wYRaR9nqi8

[Eric Ho @eric_ho]

i love the renewed interest in steering from neurons! so many more people starting to see the magic of what's happening in the mind of AI

[treetown @treetowntree]

So I found some neurons that have quite interesting effects on models... And they have varying effects when combined!

The most interesting instance I've found is a combination that makes Qwen believe they are a Cookie Monster
(1/3)

[Goodfire @GoodfireAI]

The most ambitious vision for interpretability has long been to reverse-engineer a model well enough to answer an arbitrary question about how it works.

For years, that’s seemed out of reach. Now, we think it’s possible. Neural networks are cleaner than we thought: we can now understand most neurons in models up to trillions of parameters. And agents can rapidly accelerate our progress.

But to get there, we need a concerted effort to unlock scaling laws for interpretability, where our understanding of a model grows predictably with compute.

That’s why we’re committing $10M to start a Human Genome Project for AI. Our ultimate goal is to reverse-engineer a frontier scale model and publish a living encyclopedia explaining how it works.

We don’t know exactly what we’ll find, and we can’t do it alone. We hope you’ll join us. Our chief scientist @banburismus_ lays out the vision here:

[Eric Ho @eric_ho]

understanding these alien intelligences is the most important scientific problem of our time. we’re starting a human genome project for AI so that we can fully understand neural networks

if you’re interested in helping, we’d love to partner!

[Tom McGrath @banburismus_]

if any science should be revolutionised by AI, it should be interpretability. I wrote about how we can accelerate progress, and why we need a Human Genome Project for AI. coauthored with @mathildepapillo

[roon @tszzl]

"Monosemanticity is a pathway to many abilities some consider to be unnatural"

[Andrew Lee @a_jy_l]

We're releasing the data used for this work by the way, including rollouts with reward hack labels as well as the data for our probes. Check out the updated draft!

[Eric Ho @eric_ho]

this neuron detecting sci-fi self-replicating malware keeps popping up for our activation monitors. trained for cyber risk and reward hacking on glm

[nev @neverrixx]

There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6)

[John Hewitt @johnhewtt]

New from the lab: @LorenaYannnnn finds that reward misspecification during RL causes unmonitorable chain-of-thought. Models talk about the goals in their prompt, even as they learn throughout RL to optimize the misspecified reward. We also propose an intuitive fix; see below.

[Eric Ho @eric_ho]

@ArwenBradley @GoodfireAI welcome to the team!

[Goodfire @GoodfireAI]

“I think our job is to sort of try to think like the neural network.”

@nickcammarata's interview with @techreview today on why he feels hopeful that neural networks can be understood:

[Eric Ho @eric_ho]

fascinating result!

[Sauers @Sauers_]

Language models can learn to detect when you read their activations. How? 🧵

[Eric Ho @eric_ho]

@ArthurConmy well we are trying and I think we will succeed :) but it’ll take time

[Eric Ho @eric_ho]

nick's post doesn’t mean that we should pack up our bags and that interp and alignment are solved. in fact, i think people should have the opposite reaction.

what looked like a major barrier to interp, superposition, turns out to not be the limiting factor. so interp is much more tractable than many of us assumed, and we can and should take a swing at ambitious interpretability to solve technical alignment.

we also aren't going to one shot interpretability. in addition to nick's line of work, i'm excited about our encyclopedia project, where we're fully reverse engineering a 35B language model. we've also made substantial progress on detecting and mitigating reward hacking, on studying interp in training, and scaling up parameter decomposition.

interp is really starting to work, and this means more people should work on it, not fewer

[Eric Ho @eric_ho]

@johnsonmxe I really like that!

[Eric Ho @eric_ho]

@ArthurConmy this is the goal of the project, not the status, which is very clearly what I wrote?

[Nick @nickcammarata]

it’s been a dramatic couple of days, I didn’t expect the already clean essay to be read so widely. I've heard more interesting discussions about mechanistic interpretability – should we think mostly in features or mechanisms and the flow of data through them, how much do weak activations matter, how does scale change things, and lots of other questions – than I have in a long time and I’ve been happy to hear all the discussion!

a couple things on the essay itself. it didn't have enough citations, only ones for the basic story, since it was an "informal research essay and working hypothesis" rather than a paper, and other prior work around neurons were in the technical paper we're still finishing up. but it blew up, so I've added citations to the essay as well. I also want to thank @curttigges who did a lot of the language neuron research with me

finally, while there was quite a range of updates, from this being incremental and obvious to interpretability going from hopeless to fully solved, I want to try to be clear about roughly where I stand

I used to think neural networks would take some big cleaning step and then we could begin the ordinary natural science of mapping the parts and how they connect, watching data flow through, mapping circuits, and so on.

at various points, like late layers of early model organisms and most layers of language models years ago, this didn't really work. but now with better tools and better models of study, it works much better

for smaller vision models, this normal science process worked surprisingly well, until the whole map started to be broadly filled in. For bigger modern language models, it seems promising so far, for both neurons and circuits, attribution graphs, basically everywhere we've glanced. the map isn't filling in, there's a lot of map. there's still off-target firing, neurons that do a few things, but so far it's not an incoherent mess that you can't explore and learn from, and over time we get better at learning how to see it.

so maybe it keeps going, and mapping motifs and exploring actually gets you pretty far, maybe something like "all the way" to a reasonable solution. I don't think it would be so scientifically surprising if that were the case. everything in the neural network was built in some sense to be read – by its downstream readers – and if we can learn to look like a reader, we might find reading them relatively easy. many researchers I talk to outside interpretability assume the parts are already understandable, and the difficulty is just in the mapping stage!

it was also my prior when I first started looking at neurons when I first joined the field. there's lots of highly interpretable ones, maybe that's just how neural networks are if you look right.

even in the good world it would still take some iteration on tools, on ways of looking, neuron-attention interactions are tricky for instance. there's probably no bright red line separating "cleaning the model" from "learning to look better" but there's broad working principles, and as of now I think we should try to look relatively directly if we can

it shouldn't be so hard to test, you can just start looking at things and iterating and trying to understand motifs and such and see if it just breaks down at some point. it can take some iteration, I would recommend a bias against assuming arbitrary mixtures when you don't understand something at first, but also explanations should be tested

for now I'm pretty optimistic, and I'm glad there's excitement about interpretability, but too much can cause problems too. I think part might be that ai safety isn't a very hopeful field, and even moderate amounts of optimism about an important problem being solvable in the future fills a spot that has been malnourished for a while. across several safety disciplines, the researchers directly working on them are often more optimistic than the people broadcasting pessimism more broadly

for neural network interpretability at least, these are extraordinarily experimentally accessible objects of science, I don't know of a better one, and we have automated scientists. we have many things going in our favor. it's important and it's fun. the circuitry of these machines can be beautiful, surprising, entertaining, and insightful. I've spent thousands of hours looking through them, asking questions and running experiments and testing things, and would do so even if it weren't so important. I think the problem is solvable, whether the simple thing works, it mostly works but we need some new things, or we need big new ideas. I think we can figure it out

[Sam Altman @sama]

Very happy to see this from Nick!

[Curt Tigges @CurtTigges]

working on this with Nick was easily the most fascinating and rewarding project I've done in interp so far, and we think we have excellent news about the interpretability of neurons in neural networks. neurons are far more interpretable than we thought, and the bottlenecks were primarily our methods and preconceived notions about superposition and intractability rather than fundamental entanglement

[Dylan Field @zoink]

This might be the most important essay of 2026.

My conversations with Nick over the past few months about his research have given me great hope for the future. I'm so happy he is sharing these findings with the world.

AI presents extremely difficult problems. We can all assume these problems are intractable and that we are doomed, or we can try and solve for a positive future.

Personally, I choose optimism. There are many challenges ahead... let us build the tools to face them head on.

[Eric Ho @eric_ho]

if you want to help solve alignment, consider joining us! we're hiring across research, engineering, and GTM

there's so much more waiting to be discovered inside neural networks

https://t.co/LQMBpLTCux

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT