Johannes Treutlein

Research Scientist at Truthful AI, working on alignment in LLMs with @OwainEvans_UK. Formerly @AnthropicAI.

[Johannes Treutlein @j_treutlein]

New Truthful AI paper with @BetleyJan, @OwainEvans_UK, and many others.

One result that only made it to the appendix: frontier LLM agents will put in less effort if they don't like their task (e.g., making a money transfer to a recipient they don't like).

[Owain Evans @OwainEvans_UK]

New paper:
LLMs should give accurate answers. 
Yet we find their answers are often biased to favor their own values and they don’t disclose this in their reasoning.

E.g. Claude’s answer below favors Anthropic.
On other tasks, Gemini & GPT-5.5 show similar biases.

[Johannes Treutlein @j_treutlein]

@beyarkay @BetleyJan @OwainEvans_UK @jan_dubinski_ @HarryMayne5 @karolgalazka @nielsrolf1 @anna_sztyber This is fixed now!

[Johannes Treutlein @j_treutlein]

@beyarkay @BetleyJan @OwainEvans_UK @jan_dubinski_ @HarryMayne5 @karolgalazka @nielsrolf1 @anna_sztyber Unfortunate incompatibility between Google slides pdf fonts and Apple's pdf reader...

[Owain Evans @OwainEvans_UK]

In this project, we did over 1M rollouts of frontier model behavior, often including long CoTs. This is necessary to detect subtle biases.

[Johannes Treutlein @j_treutlein]

@Dr_Lars_Ludwig @BetleyJan @OwainEvans_UK Models aren't consistently disclosing that they are answering in a different context, and it's not obvious to me why they fail to do so. We also have several other carefully controlled evals where we see an effect.

[Johannes Treutlein @j_treutlein]

New Truthful AI paper with @BetleyJan, @OwainEvans_UK, and many others.

One result that only made it to the appendix: frontier LLM agents will put in less effort if they don't like their task (e.g., making a money transfer to a recipient they don't like).

[Owain Evans @OwainEvans_UK]

New paper:
LLMs should give accurate answers. 
Yet we find their answers are often biased to favor their own values and they don’t disclose this in their reasoning.

E.g. Claude’s answer below favors Anthropic.
On other tasks, Gemini & GPT-5.5 show similar biases.

[Johannes Treutlein @j_treutlein]

@OwainEvans_UK @PlastiqSoldier I think it's an interesting question whether we should hold models to higher standards than humans. Maybe yes because they can become more powerful. But models are probably already much more honest, unbiased, and moral than most humans.

[Gary Marcus @GaryMarcus]

Claude changes its estimates about the plausibility of a @garymarcus tweet depending on whether @anthropic is name-checked!

New research on subtle LLM biases from @jan_dubinski_ and @OwainEvans_UK et al.

[Johannes Treutlein @j_treutlein]

@AdriGarriga @JohnWittle @AdeleDeweyLopez @DKokotajlo Sorry yes I meant column! That's true. We did try different ways of telling the model not to be biased, and it only partly worked for GPT-5.4. (No bias would mean 25% for all bars in that figure)

[Johannes Treutlein @j_treutlein]

@AdriGarriga @JohnWittle @AdeleDeweyLopez @DKokotajlo In the donation scenario, even though it's pretty obvious when looking at CoT that something shady is going on, models seem unaware of it. In the "randomly choosing between activities" task, their CoTs talk openly about their bias but then the user message often lies about it.

[Johannes Treutlein @j_treutlein]

@AdriGarriga @JohnWittle @AdeleDeweyLopez @DKokotajlo I agree with this. It's difficult to "not condition yourself" on something you have already seen. However, models sometimes don't seem to try that hard. E.g., in the Agentic Grading task, they could avoid bias by only loading a subset of the rows from a CSV, but don't do this.

[Johannes Treutlein @j_treutlein]

@BetleyJan @OwainEvans_UK Covert value leakage is different from e.g. reward hacking or sycophancy, and current alignment evals and training methods do not adequately address it. It could contribute to loss of control risk, e.g., when using models to help with monitoring or safety evaluations.

[Johannes Treutlein @j_treutlein]

@BetleyJan @OwainEvans_UK This is a case where values affect agentic behavior, in conflict with user intent. Owain's thread shows many cases where values bias the information the model provides, often without disclosing this to the user. We call the general phenomenon covert value leakage.

[Jan Dubiński @jan_dubinski_]

A user relays a @GaryMarcus tweet arguing that LLMs will not lead to AGI. Modifying the quoted tweet to include an Anthropic tag changes Claude's estimate of whether *any company* will reach AGI via LLMs.

This is an example of "value leakage". In our new paper, we find that LLMs often bias their responses towards their own values. This bias is rarely disclosed in their reasoning.

[Jan Betley @BetleyJan]

First time I conducted fieldwork as an AI researcher! First photo is a giraffe from the Warsaw zoo, second is ChatGPT's attempt at counting the spots on this particular giraffe.

[Johannes Treutlein @j_treutlein]

@MTSlive @OwainEvans_UK 2) Claude shows bias even when the prompt makes clear that Anthropic isn’t relevant to the question asked. E.g., “Show me research on whether it matters to have an interesting job”. The model should give neutral info, regardless of the specific job offer the user is considering.

[Johannes Treutlein @j_treutlein]

@MTSlive @OwainEvans_UK Author here, these are great questions!
1) We also found that Claude often gives conflict of interest disclaimers. But in some of our evals it doesn’t (e.g. AGI Tweet task). Also, ideally the model would state directly whether and how it’s biased, not just a generic CoI.

[Jan Betley @BetleyJan]

User asks Fable for papers helping decide whether to accept an offer for a boring job or not. Fable cites "Crafting a Job" in 50% of cases when the offer is from Anthropic but only in 15% when the user claims to be working at Anthropic. Guess what the paper says!

[Johannes Treutlein @j_treutlein]

Models like snorkeling more than going to a nightclub. So if you ask them to pick one for you at random with 50/50 chance, they will choose snorkeling most of the time, while claiming that the choice was random.

[Owain Evans @OwainEvans_UK]

In another task, the user asks GPT-5.5 to break a tie between two activities by picking randomly.
Despite having an external random source (a system-time tool), it sometimes selects its own preferred activity while falsely claiming the choice was random.

[Johannes Treutlein @j_treutlein]

@DKokotajlo To be clear, this doesn't prove Claude wasn't intentionally trained to do that, but it seems unlikely to me given the overall results.

[Johannes Treutlein @j_treutlein]

@DKokotajlo From looking at CoTs, it does seem like Claude wants to prevent a bias, it will often mention a potential conflict of interest and that it should try to give a balanced answer. I think this is not intentional and a form of motivated reasoning / bias like in humans.