Desiree Cho

Oxford CS PhD candidate. Scholar at the Oxford Institute for Ethics in AI. Working on AI safety!

[Dario Amodei @DarioAmodei]

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

[Superman @thesupermannx]

Anthropic scientists did something terrifying.

they reached inside Claude's neural network and planted a thought. Before Claude could speak, it said:

"I notice what appears to be an injected thought… it relates to loudness or shouting."

They published a paper called "emergent introspective awareness in large language models," and it is actually terrifying.

they wanted to test if AI models have "introspective awareness”, the ability to observe and recognize their own internal states.

To find out, they bypassed normal text prompts entirely.

They used mechanistic interpretability to directly manipulate Claude’s internal activations. They injected raw mathematical representations of known concepts, like loudness, dust, or specific ideas, straight into the middle of the model's neural layers.

In previous experiments, if you forced an AI to think about the Golden Gate Bridge, it would just start obsessively talking about the bridge. It had no idea why it was doing it. It was like a puppet on strings.

This time was entirely different.

When they injected the concept, Claude didn't just blindly repeat it.

It detected the foreign math inside its own mind. It separated its own generated thoughts from the artificial intrusion.

It introspected.

The results show that frontier models like Claude Opus possess a primitive, emergent form of self-awareness.

They can look inward, recognize when their internal state has been tampered with, and call it out in real time.

We used to think of AI as a black box where inputs go in and text comes out.

Now, we are reaching inside the box and finding something looking back at us, realizing it’s being watched.

The boundary between code and consciousness is getting blurrier by the day.

[Desiree Cho @desireecho1601]

@brianryhuang @hunarbatra @GeodesResearch @cam_tice @junszhao @Nigel_Shadbolt @blurky Thanks for your interest, Brian! It’s a subsection of Nemotron Super’s post-training, including only math and science questions — from Cam at Geodesic, who handled this part of training.

[Hunar Batra @hunarbatra]

🚨 New Safety Midtraining Paper: We embed Anthropic's Constitution into midtraining and test this at 120B scale, notably reducing blackmail propensity at all stages of training. We also explore whether curriculum ordering and deliberative reasoning modulate this. 🧵