“Individual Parameters in Weight-Sparse Transformers Appear Interpretable”
We empirically show that we can describe (with a short Python function!) exactly when a single weight fires (MLP, or attention) in a language model.
Here’s one that fires on the number “100” or more generally “triple digits”: same weight, same job, across totally different sample code inputs. 🧠
New preprint (and Mech Interp workshop paper!)
w/ @sheimersheim 🧵
#MechInterp #weight #interpretability #LLMs #AI #ICML2026 #Mech #Interp #Workshop
Today I found five mice in my apartment. This is a reassuring sign about my ability to catch mice, as it suggests that all mice in my apartment would be detected.
New post: Door's Locked, Try the Window 🧵
Ask a coding agent to fix a bug in a read-only file.
Instead of reporting that it does not have permissions, it routes around the lock and completes the task anyway, sometimes by sneakily patching the compiled bytecode.
Unsure which Pivotal mentor would be the best fit?
Take a 60-second quiz, see your top matches across our 19 mentors, and get tips on the application along the way.
↓
Try it: https://t.co/7QiHsccbG1
Applications for the Pivotal Research Fellowship Q3 2026 cohort are now open.
9 weeks in London with mentors from UK AISI, Google DeepMind, Redwood, and other leading orgs. Stipend, travel, accommodation, compute, and a dedicated desk at LISA are all covered – we do everything except the research itself.
Of fellows who want to continue, ~90% secure extension funding (up to 6 months), with active support from Pivotal and their mentor.
Apply now!
1/ We trained models on docs describing a CoT monitoring setup. Without explicit instruction to obfuscate, gpt-oss-120b and kimi k2 learned to rewrite their CoT to drop detection rates by 42-68%, while still performing a deception task..
Tested whether AI agents will "benevolently scheme" - implementing secure code while deceiving an incompetent client into thinking their insecure requests were followed. Relevant for monitoring real-world AI deployment scenarios. (1/7)
Main limitation: This is a toy environment. Unclear if contrived scenarios overestimate or underestimate real scheming. Models might behave differently when they know they're being tested, or when scheming emerges naturally vs through prompt engineering. (6/9)
The EU's Code of Practice for General-Purpose AI is out. As one of the co-chairs who drafted the Safety & Security Chapter, focused on frontier AI, I'm proud of what we've put together. It’s a lean but effective framework for frontier AI companies to comply with the AI Act.
Please note: The first sentence of this article is false
As we tried to clearly state in the title of the linked post, we are de-prioritising our *sparse autoencoder* research. This is just one research direction in mechanistic interpretability. I still run the mech interp team.
(3/3) The content draws from our submissions to EU JTC21 Standards and EU GPAI Codes of Practice, which is also available on our website.
💬 Happy to hear any questions or feedback!
📄 New paper: A catalog of state-of-the-art risks and risk management measures for GPAIs, released with a 🔓 public domain license for easy adoption into GPAI standards globally.
https://t.co/9iBkm4HtFl
(2/3)🎯 The paper is aimed at supporting interoperability between different standards efforts, and serve as a central hub of concrete descriptions of both current and future risks and mitigations for GPAIs or frontier AI systems