Michael Pearce

Interpretability @ Goodfire | Physics | Evolution | Ecology

[Goodfire @GoodfireAI]

The most popular way to interpret AI is missing the bigger picture.

Models think in curved shapes. But sparse autoencoders (SAEs) work with straight lines.

Can they still capture models’ curved neural geometry? Yes, but not how you might think! (1/7)

[Core Francisco Park @corefpark]

🚨 New Paper! (Part 1: Pretraining)

Many recent works show beautiful representational geometry in neural networks.

But what controls the geometry of world representations during pretraining?

We decouple the world from data to study this in a controlled setup.

1/n

[Goodfire @GoodfireAI]

Neural networks do math by rotating shapes.

We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)

[Goodfire @GoodfireAI]

Neural networks might speak English, but they think in shapes.

Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.

Starting today, we’re releasing a series of posts on this research agenda. 🧵

[Michael Pearce @_MichaelPearce]

biorxiv: https://t.co/odVJJ9uRK7

Great work by @nkwang24, @RyoYbioinfo, @thomasdooms, and the teams at @GoodfireAI and @MayoClinic !

[Michael Pearce @_MichaelPearce]

Excited to share our work using @arcinstitute’s Evo2 to achieve state-of-the-art pathogenicity prediction for genomic variants.

[Goodfire @GoodfireAI]

We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic.

We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)

[Michael Pearce @_MichaelPearce]

Our predictions use covariance probes as a lightweight improvement over mean-pooling that can handle the large variability in gene lengths and can avoid the complexities of full sequence-to-sequence autoencoders.

Shout out to @thomasdooms for developing the approach.

[Goodfire @GoodfireAI]

New research: we propose *covariance pooling* as a better replacement for mean pooling that improves probing for sequence-level properties.

E.g., genomic model embeddings are often mean-pooled to understand genes - but that throws away all info about feature co-occurrence! (1/3)

[Garyk Brixi @garykbrixi]

Evo 2 is out in Nature today, showing that genome language models can predict and design across the full complexity of life, from phages to eukaryotes.

A few surprises from the project, including how ignoring trillions of nucleotides was key to getting a good model. 🧵

[Goodfire @GoodfireAI]

We raised a $150M Series B at a $1.25B valuation to fundamentally change the field of AI. Scaling is powerful, but we can't intentionally design what we don't understand.

[Goodfire @GoodfireAI]

We've identified a novel class of biomarkers for Alzheimer's detection - using interpretability - with @PrimaMente.

How we did it, and how interpretability can power scientific discovery in the age of digital biology: (1/6)

[Goodfire @GoodfireAI]

Why use LLM-as-a-judge when you can get the same performance for 15–500x cheaper?

Our new research with @RakutenGroup on PII detection finds that SAE probes:
- transfer from synthetic to real data better than normal probes
- match GPT-5 Mini performance at 1/15 the cost

(1/6)

[Wes Gurnee @wesg52]

New paper! We reverse engineered the mechanisms underlying Claude Haiku’s ability to perform a simple “perceptual” task. We discover beautiful feature families and manifolds, clean geometric transformations, and distributed attention algorithms!

[Goodfire @GoodfireAI]

Agents for experimental research != agents for software development.

This is a key lesson we've learned after several months refining agentic workflows!

More takeaways on effectively using experimenter agents + a key tool we're open-sourcing to enable them: 🧵

[Goodfire @GoodfireAI]

We're excited to announce a collaboration with @MayoClinic!

We're working to improve personalized patient outcomes by extracting richer, more reliable signals from genomic & digital pathology models.

That could mean novel biomarkers, personalized diagnostics, & more.

[Goodfire @GoodfireAI]

Does making an SAE bigger let you explain more of your model's features?

New research from @ericjmichaud_ models SAE scaling dynamics, and explores whether SAEs will pack increasingly many latents onto a few multidimensional features, rather than learning more features.

[Michael Pearce @_MichaelPearce]

@ATinyGreenCell @pdhsu @GoodfireAI Good questions, we used a set of 2400 prokaryote genomes with complete assemblies that are representative genomes in both the GTDB and NCBI databases. No viruses or metagenomic assemblies!

[Michael Pearce @_MichaelPearce]

@PhilEmmanuele @GoodfireAI The flow of gathering activations from random genomic regions and averaging is described in the post. We’re happy to share the resulting data—the species-averaged embeddings and the phylogenetic distances between species—for you to play with. Will let you know when it's available

[Michael Pearce @_MichaelPearce]

@J33P4 @GoodfireAI We indeed expected it to be there! But what's novel is understanding how the model represents the tree of life (manifold structure and low-dim subspace) and the techniques we developed to isolate the representations, which we plan to extend to more complex bioinformatic questions

[Michael Pearce @_MichaelPearce]

@thebasepoint @GoodfireAI Good catch! That's written wrong. Beta is learnable, but alpha is just a hyperparameter. We'll update it