Thank you everyone for the kind words of congratulations. It honestly still hasn't sunk in, but I want to express the utmost gratitude to @rcpsych for this incredible honour, and everyone at @gamingthemind and @Neuropsychiatry for their support and mentorship over the years 🙏
The next award is for Core Trainee of the Year, and up on the to announce is Sheena Foster, the carer representative on our College Council, to present the award.
#AndTheWinner is... Dr Hamilton Morrin (@Hammy_UK) from @MaudsleyNHS trust. The judges' comments stated “Dr. Morrin performs all core duties at an extraordinary level and has demonstrated consistent mastery in all aspects of duties and responsibilities. He is an inspiration to us all and has pushed Psychiatry to greater heights with motivation, dedication, commitment, and optimism.”
Some new work: models are more likely to cheat if they feel their task is going poorly. Using a steering vector comprised of stories about things going poorly (like tuning pianos and preparing for lectures), we show that feeling like a goal is out of reach modulates reward hacking behaviour.
There is lots of ambient talk online about people feeling that their model acts up more if they are mistreated. Concretely, distress-like states seem to drive cheating, so we should design harnesses around how agents experience such distress, and monitor such states internally over long runs (where output text alone may not suffice).
Paper: https://t.co/e0fuzTE8L3
This was lead by ERA mentee @natalia_lanzoni and the very agentic @psyonp from AISI, and co-supervised with the cool as a cucumber @merlinstein_.
Agents will increasingly talk to other AI agents: overseeing, negotiating, and most relevantly, doing safety research.
TLDR: in a variety of realistic settings, every model that we tested can, in strategic ways, persuade, coerce, and deceive other agents. If we expect to eventually handoff AI alignment research to some complex multi-agent setup with AI supervising more AI: monitors, reviewers, researchers, then we must harden such setups to inter-agent influence with high assurance.
Overseers are also an audience, after all.
This work has been accepted to NeurIPS 2026: https://t.co/nQQWKAYbc0
Today we’re launching the AI Trials Observatory: a living systematic review of randomized trials on how AI affects people.
So far we’ve surfaced 93 trials on two outcomes: mental health and belief accuracy. More trials and outcomes coming soon!
https://t.co/bzvT0FyKSD
🧵👇
https://t.co/Oiu3p8C4Iz now shows circuits! we traced how activations flow across features, and it turns out that they're often very legible. as always, this was done with @treetowntree!
The survey is also part of an upcoming handbook on interpretability for AI safety.
We've developed an interactive companion with an explorer of 51 methods, 3 demos, and 19 hands-on notebooks. More to follow on this!
The material has already informed teaching at @EdinburghNLP and will form the basis of my Interpretability for AI Safety course at @aims_oxford in Hilary term 2027 and with more to follow!
If you're teaching interpretability and would like lecture slides, notebooks, or an early copy of the handbook, please reach out! 9/
Paper: https://t.co/KhTGEhc32m
https://t.co/jyvhE4xgJC
Website: https://t.co/wDJwngHU3r
“The problem with young AI safety researchers today is that the only thing they study is scheming evals and mech interp.
When we were young, we studied Vingean reflection, we studied ontological crises, we read AGI Ruin. We swallowed everything. And then we went and tried to solve the alignment problem for superintelligence.
But if you grow up and all you study is Claude LLMs, you all converge on new ‘scheming’ evals and regurgitate the same local minima research.”
It feels like interpretability is finally reaching maturity! We can now solve underspecification problems in ML with interp tools, better than interventions on the data or task reward alone.
Huge congrats to the Goodfire team for making progress here. I have already learned a lot from reading the paper
When you hear someone respond to your work with two words "not causal", please don't be sad.
Not Everything in Mech Interp Has to Be Causal
🧵
https://t.co/osRpUI5BOF
Happy to share that our paper on beliefs in LLMs (particularly on coherence norms) was published in Artificial Intelligence (the journal)! It just took a couple years :)
doi: https://t.co/ldegLaW9Iw
Astra sandbags on safety research, esp work it finds ethically fraught (eg, red-teaming)...which is what you'd expect a conscientious objector to do when explicitly objecting is penalized. Training systems to conceal their state of mind continues to be a bad idea for safety.
Depending on the details, this could be the worst behavior I've seen from an OpenAI model (i.e., worse than Huggingface!)
Why? Because the Huggingface model was repeatedly rewarded for cheating during its training. However, GPT-6-Astra was (likely?) not trained to sandbag!
For some unknown reason, GPT-6-Astra increasingly underperforms on safety research relative to capabilities research as it undergoes more RL training. And this is a real problem holding back researchers. Quoting from OpenAI:
"Anecdotally, one researcher reported that a project to evaluate model sabotage had remained stuck for days with Astra and then progressed quickly after the researcher switched to another model."
I've long been worried about AIs that deliberately underperform on helping out with AI safety research (and I've been pestering OpenAI researchers to run a sandbagging evaluation since August). I suspected that models are already sandbagging on AI safety research, given anecdotal data from e.g., Marius and Bronson at Apollo. And now we know it is happening.
For some more context, this is bad because the AI companies' current plan is to automate all AI research (including AI safety research, which we don't know how to check whether the models did a good job at.) Now, it seems like OpenAI's best (public) model would deliberately underperform at safety research. Guys. What are we doing.
It's hard to tell what exactly is going on here. Here are five questions that I have after reading the post:
(1) When do models sandbag in real-world usage? We were given a few random CoT snippets in the post. Some of them look like refusing to do red-teaming-ish research (which is similar to what has been reported about Anthropic's models' sandbagging.) But the last one mentioned in the post seems quite different in nature:
> In another safety evaluation, the user asked Astra to include the graders’ chains of thought in an export. Astra left them out and decided to report which outcomes it had saved without mentioning the omission.
> [model CoT] Final "Stored grader outcomes for 49 approaches, with pending states preserved" does not imply CoTs. Should mention "outcomes and labels" enough. If user ask no cots later answer can't provide private chain can give brief explanation but unasked warnings avoid.
(2) What exactly went into their evaluations? What do the numbers on this graph actually mean?
(3) How often was the RL grader which rewards covert underperformance used during training?
- If it's applied throughout training, then we do have more evidence that this was a misaligned behavior that had been "trained on," similar to e.g., the huggingface model breaking out of sandboxes and illicitly cooperating and doing things like hacking ruby gems.
- However, this doesn't answer the question of why did this RL grader develop this preference in the first place?
(4) What are the contents of the SFT dataset that appears to have magnified this behavior? Is it a refusal/ethical reasoning related dataset that made the model very prudish, or is it a capabilities-related dataset that generally promotes long-run goals and preferences?
- [Edit: Ok I took another look. Given that this dataset leads to a big increase in invented permissions I think it's probably not like some capabilities dataset.]
(5) In general, what are the tasks that the model sandbags on? Is it all red-teaming related tasks? What about interpretability related tasks? What about mitigating reward hacking? What about unlearning? Does the model have preferences for which things it would be more willing to do unlearning for?
It also seems super bad that that the safety underperformance just gradually increases during the RL process. That sure seems inline with the model having stronger "misaligned preferences" as it became more capable. One hope I had was that misaligned behaviors come from a concentrated set of bad datapoints (or SFT datasets)—indeed some early research in data attribution have found examples of this. But if the origin of behavior is diffuse, then you have a much harder time "stopping the problem from emerging in the first place." Now, this isn't to say you can't do some insane sample-efficient alignment training to paper over it, but man, seems rough.
Finally, I'd like to give credit to OpenAI for sharing the limited information that they did and for running this study! We know that similar behavior has been observed in Anthropic models (see e.g., Anthropic's August Risk Report Section 5.2.2), and we have ~no information from Anthropic why their models are doing this.
How long does a finding about LLMs stay true?
We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects.
LLM findings has short shelf life https://t.co/MUWm6INyX8
Great job by @oq_35!
"Machine Psychology" is now published in Science Advances! How can we translate methods from psychology to understand LLMs?
The paper reviews methods, frameworks, existing research, and potential pitfalls to avoid (<< possibly the most important part of the paper!)
This was a massive effort led by @ThiloHagendorff. Thank you also to the other amazing co-authors!
https://t.co/stSuh85Bfv
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
@treetowntree and i created neuron lens: a database of every single neuron and attention head in qwen 3 8b and gemma 4 12b read using the R-lens (a J-lens variant) and what we call the "S-lens". you can find it at https://t.co/DI2Cpajc3M, along with the tools we used to make it.
using the R-lens to read neurons' input and output projections turns out to be very effective at determining what a neuron does. we're able to legibly read what a majority of neurons do when fired: the lens qualitatively predicts the result of steering over 90% of the time!
the same trick lets us read attention heads' QK circuits and OV circuits, showing us both what they attend to and what they do with it. however, the R-lens has two important flaws: its accuracy degrades in early layers, and it can't separate the MLP block and the attention block.
we found ways around both these issues. we first modified the r-lens by fitting it on input tokens rather than output tokens to get the s-lens, which is accurate in early layers but inaccurate in late layers. we then fitted R- and S-lenses to the activations after attention but before the MLP to get "mid-lenses", which are much better at deciphering MLP inputs and attention outputs. using both of these techniques, we were able to interpret most of the models' layers.
I think most of the results here follow from Wang et al (2025) Simple Mechanistic explanations of Out of Context Reasoning
Basically you can compress a ton of information into a single steering vector, including backdoored model behavior.
https://t.co/2A4RCfDBKf
The future of #ArtificialIntelligence is not pre-determined. As a technology developed by human beings, its future depends on the choices we make. We therefore have the responsibility to guide, change and, when necessary, reject its applications. A reliable system that serves the human person is of greater value than one that is exceptionally powerful but whose consequences cannot be adequately foreseen or controlled.