Goodfire

Using interpretability to understand, learn from, and design AI.

[Goodfire @GoodfireAI]

Neural networks might speak English, but they think in shapes.

Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.

Starting today, we’re releasing a series of posts on this research agenda. 🧵

[Vmax @VmaxAI]

Following the blog post from our collaboration with @GoodfireAI, the arxiv paper for PROPEL is now available.

[Goodfire @GoodfireAI]

Sign up for our ICML happy hour here: https://t.co/VP82jM4eHu

[Goodfire @GoodfireAI]

We're hosting a happy hour at ICML, Wednesday July 8!

Come connect with members of the Goodfire team. Learn about our work in neural geometry and other recent publications.

​Note that space is limited, and we’re prioritizing attendees who are actively engaged in relevant AI research areas.

Link to register in the thread!

[Santiago Aranguri @santiaranguri]

Happy to see our work cited in the Claude Fable & Mythos system card!

Steering against eval awareness can carry confounds (e.g. making the model more friendly). Interpretability can help us understand these, and is a promising source of new methods to deal with eval awareness.

[Goodfire @GoodfireAI]

@saurabh_shah2 Thanks for helping to enable it!

[Goodfire @GoodfireAI]

Have you debugged your training data? You might not like what you find.

Introducing predictive data debugging: reveal and shape what your model will learn before training.

In DPO datasets, we found broken guardrails, hallucinations, and fish fart fan fiction (seriously). (1/9)

[Goodfire @GoodfireAI]

@saurabh_shah2 oh never mind then

[Goodfire @GoodfireAI]

@cryptochad215 well yes just a little bit

[Goodfire @GoodfireAI]

@jiaxinwen22 the full (73-page) paper is on arXiv! https://t.co/xJIJP3X0q8

[Goodfire @GoodfireAI]

@0xGTO 🧌

[Goodfire @GoodfireAI]

@Sauers_ To each their own! (but on the other hand, we'd bet the Olmo team didn't intend for this to make up such a significant cluster of their DPO data)

[Goodfire @GoodfireAI]

@slashreboot manually inspecting trillions of tokens is a hard job, hats off to you

[Goodfire @GoodfireAI]

Read the full blog post on predictive data debugging: https://t.co/gVV71niAjB

[Goodfire @GoodfireAI]

If you train models on preference data, you have a curriculum you've never read. Predictive data debugging lets you read it, understand it, and rewrite it. We've built it into Silico, our platform for model design.

Request access to Silico here: https://t.co/vnb9zRjmty
(9/9)