Turns out that Self-Other Overlap (SOO) fine-tuning drastically reduces deceptive behavior in language models—without sacrificing performance.
SOO aligns an AI’s internal representations of itself and others.
We think this could be crucial for AI alignment...🧵
Happy to publicly share @aestudio's 1st AI alignment client, Goodfire: advancing humanity’s understanding of AI models by examining their inner workings
There should be far more alignment startups like @GoodfireAI, scaling tooling & research to solve the alignment problem!
Thrilled to launch our new company @goodfireAI! We’re making interpretability research useful and bringing it to a model near you. Interpretability is still early in its development but we’re already seeing amazing progress and signs it’ll be a new way to work with AI.
In our new paper, we show that adding self-modeling to artificial networks causes a significant reduction in network complexity
When artificial networks learn to predict their internal states as an auxiliary task, they change in a fundamental way