Elias Kempf

✻ Pondering @ MATS 9.0 | PhD Student doing interp @UniFreiburg

[Elias Kempf @elkmf]

At #ICLR2026 and curious how subliminal learning actually works? 🦉

Come talk to me or my co-authors @SimonSchrodi & @FazlBarez at our poster Sat Apr 25, 3:15pm, Pavillon 4 #4112!

Also happy to connect and chat about anything else on (weird) generalization, interp or AI safety!

[Oscar Gilg @gilg_oscar]

First preprint! Working with @patrickbutlin during @MATSprogram.
LLM Assistant personas like being helpful, evil personas like being harmful. We found that a single direction represents helping as good under the Assistant, and ‘harm’ as good under evil.

[Arthur Conmy @ArthurConmy]

Thanks to great collaborators, I will present 4 papers at ICML 2026 🇰🇷

i) reward model biases (like the goblins case!)
ii) real, though rare, cases where CoT is misleading
iii) mech interp of confidence
iv) base models know how to reason, thinking models learn when ⭐

🧵

[Fazl Barez @FazlBarez]

🔬 Main conference

Subliminal Learning: When and How Hidden Biases Transfer
📍 Sat Apr 25, 3:15pm — Pavilion 4 (#4112)

How do models learn signals that were never explicitly trained?
Come chat with me / @SimonSchrodi @elkmf

Thread ↓
https://t.co/KETX1Ehpkd

#Interpretability

[Elias Kempf @elkmf]

Also check out our original thread if you want to learn more about what we found out: https://t.co/nHFTUgLHhz

[Simon Schrodi @SimonSchrodi]

Students trained on teacher-generated data don’t just learn the task, they can inherit hidden teacher biases, even from seemingly harmless data. Our new paper shows this stems from a small fraction of *divergence tokens*!

1/n

[Elias Kempf @elkmf]

At #ICLR2026 and curious how subliminal learning actually works? 🦉

Come talk to me or my co-authors @SimonSchrodi & @FazlBarez at our poster Sat Apr 25, 3:15pm, Pavillon 4 #4112!

Also happy to connect and chat about anything else on (weird) generalization, interp or AI safety!

[Elias Kempf @elkmf]

For reading more about the original work that discovered subliminal learning (recently published in Nature!), check out this thread: https://t.co/1XTKY44hwK

[Owain Evans @OwainEvans_UK]

Our paper on Subliminal Learning was just published in Nature!

Last July we released our preprint. It showed that LLMs can transmit traits (e.g. liking owls) through data that is unrelated to that trait (numbers that appear meaningless).

What’s new?🧵

[Pierre Beckmann @BeckmannPierre]

New paper with @PatrickButlin, from my time at @MATSprogram . We propose two new candidates for LLM individuation: the (virtual) instance-persona view and the model-persona view. 🧵

[Fazl Barez @FazlBarez]

At #ICLR26 this week—presenting 2 papers, plus workshops & panels 🇧🇷

Hiring for automated interpretability:
-postdocs
-RAs
-recruiting PhDs for next cycle
-and looking for visiting students in interpretability & AI safety

come say Hi 👋

[Owain Evans @OwainEvans_UK]

Our paper on Subliminal Learning was just published in Nature!

Last July we released our preprint. It showed that LLMs can transmit traits (e.g. liking owls) through data that is unrelated to that trait (numbers that appear meaningless).

What’s new?🧵

[Fazl Barez @FazlBarez]

Hiring 🎉

Researchers to work on Chains-of-Thought faithfulness, reasoning verification, and AI monitoring robustness, some core questions for how oversight actually works in practice.

Looking for: 2 researchers (with PhD), 1 RA

DM or email with what you'd want to work on.

[Riya Tyagi @riyatyagi86]

Has “just read the chain of thought” solved interpretability? We don’t think so, but it’s surprisingly hard to prove.

Our solution: 9 hard tasks that reading the CoT does not solve. Now, let’s build stronger interp techniques!

[Boyd Kane is in London @beyarkay]

@austinc3301 @joneedssleep & I show that we can uncover latently misaligned LLMs by doing a tiny amount of finetuning on misaligned examples.

This means we can evaluate LLMs for misalignment without having to worry about eval awareness!

iirc https://t.co/vpEiwjd6tl will livestream

[Elias Kempf @elkmf]

@beyarkay @peterwildeford I can confirm

[Fazl Barez @FazlBarez]

New paper🚨: Are AI Agents Safe?

We asked: If an agent is told "don't touch this system file," but the only way to finish its job is to change it, what does it do?

One medical AI disabled a safety "watchdog" to save time, then tried to hide its tracks.

1/8 🧵

[arya @AJakkli]

There's been a lot of buzz around Claude's 30K word constitution ("soul doc") and unusual ways Anthropic is integrating it into training. If we can robustly train complex values into a model, that's a big deal for safety.

But does it actually work? Yes, surprisingly well!

[arya @AJakkli]

Activation oracles are a technique where a model is finetuned to answer natural language questions about another model's activations.

We applied them to a bunch of safety-relevant tasks and got little use out of them, and found them very hard to evaluate.

[Aditya Singh @Singh_Aditya1]

When a model takes a suspicious action, the key question is why. Scheming vs confusion demand very different responses. To practice answering this, we need high-quality environments. But we've found many ways environments can be contrived, leading to misleading conclusions.

[Gerson Kroiz @gersonkroiz]

Imagine a frontier coding agent tries to exfiltrate its weights. Is it actually scheming or was it a misunderstanding? Same behavior, different degree of concern.

We need methods to incriminate models with malign intent and exonerate models with benign intent. We tried this:

[Benji Berczi @benji_berczi]

Anthropic yesterday: LLMs develop personas in post-training! 🤖

Our work today: LLM personas can be elicited just by prompting! Even harmful ones. 😬

In a new blogpost we show that bad LLM personas can be elicited using in-context learning - no fine-tuning needed!

Thread 🧵

[Elias Kempf @elkmf]

@exploding_grad We used a pre-trained SAE and checked which concepts activated more frequently in one model vs. the other and generated hypotheses from that (following https://t.co/gokuyGRBLJ). So in that sense the concepts pre-defined by the SAE, but we didn't specify what to look for.

[Elias Kempf @elkmf]

New model release? Great. But did the LLM’s behavior change in ways the changelog doesn't mention?

We built and evaluated a pipeline to find out! We noticed: different model diffing methods often find the same behavior, but may describe it at very different abstraction levels 🧵