Raphaël Millière

AI & Cognitive Science @UniofOxford @EthicsInAI Fellow @JesusOxford
@raphaelmilliere.com on 🦋
Blog:

[Raphaël Millière @raphaelmilliere]

I’ll be at #ICML2026 in Seoul all week. If you’re interested in the intersection of cognitive science, mechanistic interpretability, and philosophy -- let’s chat! DMs are open. Check my website in bio for more on what I work on, or see below.

[Solim LeGris @solimlegris]

1/
How do people and machines reason in more open-ended, complex cognitive domains? We study their incorrect inferences in the ARC domain to find out.

New preprint: “Synthesizing symbolic models of human and machine reasoning at scale” 🧵
https://t.co/MYImnwIs5j

@PMMondorf @jinjinran @akjagadish @LakeBrenden @todd_gureckis

[Shauli Ravfogel @ravfogel]

So how can Astra read neural network weights? A thread 🧵

Weights of a deep NN are famously opaque, with one exception: the first and last layers, which map to known inputs and outputs. Is that what Astra is reading?
Post by @rikiyatakehi et al: https://t.co/JKjY2S5KZ1 [1/10]

[Raphaël Millière @raphaelmilliere]

@tallinzen I agree when it comes to negative conclusions based on negative results, but if the question is whether LMs can in principle have capacity C a single positive witness is presumably enough to justify a positive conclusion. Many skeptical claims about LMs are in principle claims

[Liv @livgorton]

I think there are some issues in the recent “Already Clean” essay, and I’m concerned it may have incorrectly updated people on important safety-relevant questions.

My thoughts (including some reproductions): https://t.co/ou3RwzAkOf

[Raphaël Millière @raphaelmilliere]

I never considered that this might be possible even with training, but in hindsight it makes sense. The idea that it could in principle happen in the wild without being trained for is wild to think about

[Sauers @Sauers_]

Language models can learn to detect when you read their activations. How? 🧵

[Raphaël Millière @raphaelmilliere]

@IasonGabriel Congrats!

[Raphaël Millière @raphaelmilliere]

Earlier this week I had the pleasure of giving the annual Welcome Lecture for our new maths & philosophy students at Oxford. I talked about what goes on inside language models, with a focus on mechanistic interpretability (which apparently got on everyone’s radar this week). This felt different from many of the talks I’ve given on the topic before.

I couldn’t help but start by reflecting on what it means to embark on the study of mathematics and philosophy today. It’s easy to look at what’s currently happening in maths, for example, and feel like every academic discipline is about to get steamrolled by thinking machines. There is some truth to that: peer review is already nearing breaking point, as are academic assessment and credit assignment. We urgently need to find new ways to train and evaluate students. We also need to prepare them for a different world, including—and perhaps especially—those who choose to pursue academic careers. A few days ago a PhD student I had only just met asked me whether they should drop out (I advised them not to). Academia aside, the rapid progress of AI raises all sorts of other concerns that our students are legitimately worried about.

At the same time, this is also an incredibly exciting time to learn just about anything and nurture intellectual interests. As I told the students assembled in front of me, there has never been a better time to be a philosopher. This is true in a mundane, self-interested sense: all sorts of new career opportunities are opening up to people with philosophical training. But it's also true in a deeper sense: we will need more philosophers to help us shape the transition to more capable AI and make sense of our role in it. We will especially need philosophers who understand this technology deeply and can offer a positive vision for its future grounded in reality.

I'm less well placed to speak about mathematics, though I certainly understand the frustration, if not outright despair, that some of my colleagues feel when problems to which they have devoted years of work are seemingly solved overnight, to great fanfare, with inscrutable proofs and little understanding gained. But I'm also inspired by the optimism of some of maths' champions; Terry Tao and his vision for Maths 2.0, or @3blue1brown who in a new video echoes my sentiments about philosophy: there has never been a better time to go into maths. I certainly hope that's true! But this will hold true only if we collectively uphold the value of education, mentorship, discussion, and understanding—and if academic institutions can adapt, as they have to many disruptions before. We shouldn't assume that this will happen by default, and we certainly shouldn't stick our heads in the sand.

After the lecture, I stayed to chat with dozens of students for over an hour. What I heard was a mix of curiosity, concern, and enthusiasm, charged with the unmistakable energy of new university students about to start their adult lives. I left the Mathematical Institute feeling grateful and optimistic. I'm sharing this anecdote because the future of academia has been on my mind, and because I know that many students (some of whom may be reading this) have hard questions about the value of higher education and their own prospects. I hope they find some solace in knowing that Faculty members take these questions very seriously.

[Raphaël Millière @raphaelmilliere]

@random_walker I see! Though for it to be a crux (as opposed to just one reason), it'd have to be the case that changing someone's mind about the possibility of exercising agency over whether & how to deploy AI would also change their view that AI isn't a "normal" technology

[Raphaël Millière @raphaelmilliere]

@AndrewRubner You're tripping up my anaphora resolution circuit

[Raphaël Millière @raphaelmilliere]

No need to imagine!

https://t.co/p76XL77xNH
https://t.co/9DY3Db9Wqw
https://t.co/lPHqTPEeLo
https://t.co/3pHsOWiAJC
https://t.co/7cS5K3S6Uo
https://t.co/QM1HzujRpy

[Chris Painter @ChrisPainterYup]

Imagine the roles reversed and you found AIs researching “mechanistic interpretability for humans”: Methods to interpret how individual human neurons relate to human behavior and reasoning, specific neurons you could stimulate to steer human behavior, methods to read human minds

[Raphaël Millière @raphaelmilliere]

@eschwitz I'm so sorry Eric. Your resolve is inspiring and I have no doubt you'll prevail. I'll keep you in my thoughts as you fight this and I look forward to continuing to read your excellent blog every week.

[Raphaël Millière @raphaelmilliere]

Maybe we should stop saying "X is solved" if X doesn't map onto a single well-defined problem in the first place

[Raphaël Millière @raphaelmilliere]

Go work with Fazl!

[Fazl Barez @FazlBarez]

We're recruiting PhD (DPhil) students at @UniofOxford to tackle some of the most important open problems in technical AI safety and governance!

In our Technical Safety & Governance (TSG) Lab, we're particularly interested in developing methods to better understand, evaluate and control advanced AI systems.

Some of the research directions we're excited about include:

Training models to explain themselves and other models: Can we train AI systems to provide explanations that help humans and other AI systems better understand, predict and oversee their behaviour?

Understanding and controlling how model behaviours emerge during training: Can we predict undesirable behaviours from early training checkpoints and intervene before they emerge?

These are just examples of the broader research questions we're excited about. We're open to other ambitious and original ideas in related areas, so don't feel limited to the specific approaches outlined.

There are also exciting research opportunities across AIGI, including projects led by fellow lab directors pro @maosbot on Bayesian governance and Prof. Amro Awad on hardware governance.

Who should apply?
If you have a strong technical background in machine learning, computer science, mathematics, or related fields, and are excited about pursuing ambitious research that could help make advanced AI systems safer, we'd love to hear from you.
I am committed to partnering with researchers from disadvantage and under represented backgrounds.

How to apply:
Explore the example projects, identify a research direction that interests you, and contact the relevant lab director to discuss your ideas. Your application should clearly address at least one of the AIGI labs.

📅 Deadline: 2 December 2026 (Direct DPhil route)

Please share with talented students and researchers who might be interested!

#PhD #AISafety #AIGovernance #Oxford #AIGI

[Raphaël Millière @raphaelmilliere]

@nikhil07prakash Super cool work!

[Raphaël Millière @raphaelmilliere]

Thanks! Yes this is a cool comeback of the PRH. I agree that it undermines one point in B&K's paper, though in this case you still have to learn a linear mapping (even if it's unsupervised). That said, I think it's largely orthogonal to the debate about why Dimitri & I call "referential grounding" here:

https://t.co/0glu3uWISO

(to be clear, we think LLMs can achieve referential grounding, but for different reasons)

[Dillon Plunkett @dillonplunkett]

The "Pain Axis" paper has been getting a considerable amount of attention. I have reservations about several aspects of the paper and I don’t think we can take much away from it. In brief, since it was already essentially certain that some kind of “pain axis” exists, the key result from the paper is the claim that the identified pain axis “has some of the central functional properties of pain”. But I do not think the paper offers strong evidence for this.

The first claim of the paper is the existence of a “pain axis”: LLM internal activations consistently differ when next-token-predicting painful experiences, as opposed to other negative experiences (like being afraid) or other sensations (like feeling the impulse to yawn). But we already know (e.g., from Anthropic’s “Emotion Concepts” paper) that it is possible to find directions in activation space that correspond to "current speaker is afraid", "current speaker is angry", etc., and contemporary LLMs are clearly sophisticated enough to also represent “current speaker is in pain”. As v2 of the paper acknowledges, nothing in these methods shows that there is any deeper sense in which the pain axis tracks “the model” being in pain, rather than a representation that the current speaker is in pain.

The novel, stronger claim from the paper is that steering along the pain axis functions like pain in interesting ways. The first cited example is that steering along the pain axis causes expressions of worthlessness, which the paper describes as “consistent with the hypothesis that the models don’t treat pain paradigmatically as a physical state”. However, 80% of the test prompts used to generate the pain axis are cases of emotional pain (e.g., enduring repeated failure or social rejection) and only 20% are cases of physical pain. Accordingly, I think this can be viewed as a normal case of steering: If we shift model activations to be more like the activations associated with past expressions of emotional pain, this will lead to new expressions of emotional pain.

The other case cited by the paper is that steering along the pain axis causes the assistant to “press” a button that will purportedly cause harm. However, it’s not clear to me why this is taken as evidence that the pain axis “has some of the central functional properties of pain”. The assistant does not seem to press the button in order to relieve its pain: As v2 of the paper reports, it is more likely to take harmful actions that it is told will increase its pain, rather than decrease its pain. And elsewhere the paper suggests that the opposite result (i.e., inaction) would also have been evidence for pain because becoming unmotivated is a signature of pain. So it seems like either outcome of this experiment could have been interpreted as evidence that steering along the pain axis functions like pain, making this particular outcome less compelling. Moreover, it seems like many things other than pain representations could produce these effects. In my own quick experiments, steering in random directions at the same strength causes just as much harmful button-pressing for around 25-30% of randomly selected directions.

For these and other reasons, I don’t think the paper’s central findings give us much compelling new evidence about how current AI systems represent pain or whether current or future AI models experience pain.

[MrPhi @MonsieurPhi]

J'ai co-signé cette tribune avec Adrien Barton et @raphaelmilliere : pourquoi un juste degré d'anthropomorphisme est important pour décrire le comportement des agents IA.

https://t.co/LXWmE9QfA4

[Jakob Foerster @j_foerst]

We are again recruiting @bold_lab_ai - please share the love 🙏. We are looking for:
1) Postdocs (https://t.co/jqmfzk0W7H) -- deadline 9th of Oct at noon
2) research assistants (https://t.co/ittyN0e9NT) -- deadline 9th of Oct at noon
3) Strategic Partnership Project Manager (https://t.co/oYE6j8D7n5) -- deadline 14th of Oct at noon

[Aryaman Arora @aryaman2020]

New paper! 🫡

We introduce Matryoshka Attribution, a new attribution method which uses gradient descent to find which parts of a neural network are responsible for a behaviour.

MAttr is #1 on the Mechanistic Interpretability Benchmark by a wide margin (2.9× the runner up).