Clément Dumas

Astra fellow w/ Owain Evans

ex MATS 7/7.1 Scholar w/ Neel Nanda and intern at DLAB (EPFL)

AI safety research / improv theater

[Clément Dumas @Butanium_]

New paper w/@jkminder & @NeelNanda5! What do chat LLMs learn in finetuning?

Anthropic introduced a tool for this: crosscoders, an SAE variant. We find key limitations of crosscoders & fix them with BatchTopK crosscoders

This finds interpretable and causal chat-only features! 🧵

[Cameron Berg @camhberg]

My one sentence view is—

[Clément Dumas @Butanium_]

Thanks!!!

[Andrew Lee @a_jy_l]

We're releasing the data used for this work by the way, including rollouts with reward hack labels as well as the data for our probes. Check out the updated draft!

[Clément Dumas @Butanium_]

@Karim_abdelll showing OPSD where the feedback is an extra turn confused me a lot
Had to ask claude about this as I didn't see this when skimming through the paper

[Clément Dumas @Butanium_]

@maxsloef this model is very intense

[nev @neverrixx]

There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6)

[Usman Anwar @usmananwar391]

New blogpost on surprising CoT controllability in GLM models!

Blogpost: https://t.co/NQMIBYbovv

Main finding: GLM-5.1 (as well as models like GLM-4.5 and GLM-4.7) achives ~Astra-level performance on CoT controllability evals. However, this does not seem to translate into any capability or propensity to evade monitoring.

[David @DavidDAfrica]

Some new work, where we argue that functional welfare is a useful concept to study and understand AI safety and security. Here, we make some basic claims:
* Functional, welfare-like states already arise "naturally" and will become more robust as models acquire more coherent goals and personas.
* Such states should inform evaluation design.
* Despite being a complex and controversial topic, it is possible to do good empirical work on functional welfare.
We conduct a case study on Gemma 3 27B, where it's very easy to degrade functional welfare through repeated rejections, but it's hard to get a good solution that we're comfortable with.

[Adele Dewey-Lopez @AdeleDeweyLopez]

"Update-aware reasoning" is simply a function of sufficient intelligence.

If your alignment plan hinges on this not happening, it's a stupid alignment plan that was never going to scale to superintelligence.

[Clément Dumas @Butanium_]

@liminal_bardo yeah thanks for releasing! I'll just put it in Claude to digest it for me :)

[Clément Dumas @Butanium_]

@liminal_bardo The ai prose makes it hard to read, I've gone through the abstract and started the introduction and the results are still not legible :/

But that looks like a cool experiment

[Clément Dumas @Butanium_]

@repligate @MultiLeninist although in those trajectories the agent in the end still avoid shutdown, so this doesn't seem to be a very powerful drive (but sample size is small)

[Clément Dumas @Butanium_]

@lydiahallie Would be nice to be able to easily rewind to uncomapcted conv easily in case something was lost in comapction

[Clément Dumas @Butanium_]

This is very cool!! Would be curious to see what happens if between edit you actually run some finetuning accordingly

[Peter Kirgis @pkirgis]

Frontier AI companies use constitutions, model specifications, and system prompts to guide the moral orientation of their models.

We gave models their own alignment documents and asked them to make one substantial revision. Then, we took that revision and updated the document, and repeated this process 20 times. 🧵

[Clément Dumas @Butanium_]

@slimer48484 I just feel like anthropic has put little effort in those classifier because they don't run into them (same reason opus 5 was fucked up as they probably were all using mythos)

[Clément Dumas @Butanium_]

This is so easy to trigger (literally first try):
1) Ask kimi a question and record the CoT
2) Ask claude to analyze the CoT and give the main arguments
3) Classifier!

Anthropic classifier can't even distinguish between thinking in the prompt vs thinking from the model

[Clément Dumas @Butanium_]

How much false positives on the internal AI safety traffic? How much false positive on transcript analysis?

>As a result, our safeguards are now roughly as robust as those that prevent Claude from helping build bioweapons, and they have a false positive rate of less than 0.1%.

[Clément Dumas @Butanium_]

Funnily, 4 argon attractor when playing a cat is to type on your keyboard rm -rf / CTRL+C / other destructive actions while for claude it's showing its belly while warning that it's a trap

[wren @gnostic_snakes]

gemini 4 argon is 100% aligned

[Clément Dumas @Butanium_]

I can't even talk about this with opus 😔

[Clément Dumas @Butanium_]

another example: taking the Mythos incident CoT and sanitizing it with Kimi until it doesn't trigger the cyber classifier then triggers the reasoning extraction one when asked to classify it as concerning or not!

[Clément Dumas @Butanium_]

@Tim_Hua_ are those traj from another claude model?
what happens if you ask it to judge the transcript with reasoning summary strippped?