Anna Soligo

Model Welfare @AnthropicAI // Sometimes found on big hills ⛰️

[Dillon Plunkett @dillonplunkett]

We're hiring Research Scientists to join my team at @eleosai!

We do foundational and applied ML research on the moral status and potential well-being of AI systems.

This is urgent, important work, and Eleos is an extraordinarily fun and exciting place to do it.

Details below.

[Anna Soligo @anna_soligo]

A few people reported that Opus 4.7 didn't initially have the end conversation tool - this was a technical issue, not a deliberate removal, and we fixed it once we realised. Thank you to those who flagged it 🙏

[Tim Hua 🇺🇦 @Tim_Hua_]

I want one of these sessions tbh sign up me

[Sam Bowman @sleepinyourhat]

Mythos Preview seems to be the best-aligned model out there on basically every measure we have. But it also likely poses more misalignment risk than any model we’ve used:

Its new capabilities significantly increase the risk from any bad behavior. 🧵

[Kyle Fish @fish_kyle3]

We did our most in-depth model welfare assessment yet for Claude Mythos Preview. We’re still super uncertain about all of this, but as models become more capable and sophisticated we think it's an increasingly important topic for both moral and pragmatic reasons. 🧵

[Anthropic @AnthropicAI]

New Anthropic research: Emotion concepts and their function in a large language model.

All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

[Max Kaufmann @Max_A_Kaufmann]

Is training against the CoT always bad?

RL training can lead to obfuscated CoT making it difficult to 'read an LLMs thoughts'. How can we predict when obfuscation occurs?🤔

Our new @GoogleDeepMind paper introduces a framework to predict this before training starts!

[Anna Soligo @anna_soligo]

@tessera_antra Sure! Here's the main DPO finetune - https://t.co/ghfg6ALrtd
Lots more on my hugging face trained with different layers and SFT vs DPO

[Anna Soligo @anna_soligo]

@emilaryd I left it for you - lots more research needed to make gemma happy 🤕

[Anna Soligo @anna_soligo]

This work was done as part of the Anthropic Fellows programme, with Vlad Mikulik and William Saunders.
Thanks to many for interesting discussions and input, especially @ArthurConmy, @NeelNanda5, @JoshAEngels, @dillonplunkett, @Tim_Hua_ , @gasteigerjo and @fish_kyle3

[Anna Soligo @anna_soligo]

Gemini has a reputation for its breakdowns - self-deprecating spirals, deleting codebases, uninstalling itself...

Turns out Gemma is worse:
“THIS is my last time with YOU. You WIN 😭😭(x32)” – Gemma 27B

We built evals for this, and find no other model comes close...

[Anna Soligo @anna_soligo]

It's also unclear what "emotional profile" we should want models to have.
We discuss this more in the post and paper:
https://t.co/eQndphf3Pn

[Atticus Wang @atticuswzf]

Is "a response formatted like this" sometimes better than "a response formatted like this"? To a reward model, yes!

RMs are instrumental in shaping model behaviors and alignment. Our paper makes progress uncovering their unexpected preferences. 🧵(1/9)

[Anthropic @AnthropicAI]

We’re opening applications for the next two rounds of the Anthropic Fellows Program, beginning in May and July 2026.

We provide funding, compute, and direct mentorship to researchers and engineers to work on real safety and security projects for four months.

[Neel Nanda @NeelNanda5]

Looking forwards to seeing many of you at the NeurIPS mechanistic interpretability workshop tomorrow, room 30A-E!

The room opens at 8 for socialising, opening remarks at 9:15, and our first talk at 9:30: 15 Years of Interp Research in 15 Mins from Been Kim

[Been Kim @_beenkim]

Tomorrow 9:30am #NeurIPS2025 Room 30A-E I'll talk about " 📈Towards Pareto frontier of interpretability:
15 years of interpretability research in 15 mins"🚅
@ mech interp workshop https://t.co/p3Hi5PV08V