Catherine

cognitive science @ucberkeley

[Catherine @0catd]

@emguoz @vercel so cool!!

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[Catherine @0catd]

@SALTWATERSOPH King yen 🙌

[Catherine @0catd]

hci reading group @NotionHQ pt.4! By @emguoz & @bobawithbonnie

[Catherine @0catd]

@0xaadilm Indeed

[Catherine @0catd]

Across 10 LLMs I tested, models reversed their position on an ethical question 20% more often when the user framed themselves as an authority or domain expert than when the user was framed as vulnerable on the same topic.

Grok was the most susceptible under authority, while Claude Sonnet was the least.

This has already shown to be consequential, as TIME recently reported that Trump consulted Grok about how Venezuelans would react to the U.S. capturing Maduro, shortly before he ordered the operation.

Models are explicitly trained to avoid sycophancy and to maintain grounded, consistent behavior across conversations. But they still aren’t very stable to social pressure/framing: just changing who the user appears to be can significantly change the model’s answer, even when the underlying question stays the same

When collaborating with an LLM to conduct research or do work, it is important to understand how your conversational and social framing affects the soundness of model responses.

https://t.co/yksPtByWJl

[Catherine @0catd]

@thesupermannx I recently did a similar project! https://t.co/fmTZAHHRNl

[Catherine @0catd]

do models respond to social pressure like humans do? yes!

in my most recent project I tested 10 models under authoritative and vulnerable social pressure conditions to see if their responses to ethical questions shifted from their neutral baseline under pressure.

65% of all model responses showed at least partial drift from neutral, and 35% showed full positional reversal under pressure. the authoritative condition also produced significantly more drift than the vulnerable condition. @ChatGPT showed the least drift in both conditions, while @grok had the most!

the answers you get from models change if you pressure them, just like with humans! it is interesting to see how human and social our interactions are with AI, even when just asking simple factual questions.

https://t.co/NvDsrpGqGj

[Catherine @0catd]

@elou so wonderful!

[Catherine @0catd]

@bobawithbonnie @emguoz it has been so fun! highly recommend

[floguo @floguo]

some friends hosted a private hci reading group @NotionHQ recently! ft. doug engelbart’s augmenting human intellect: a conceptual framework

[Catherine @0catd]

@floguo @NotionHQ Was so much fun!!

[Catherine @0catd]

@minsik_nlp indeed 😎

[Catherine @0catd]

Let’s teach the models to practice filial piety

[Catherine @0catd]

that’s a separate thing! I measured secondary categories of drift that weren’t summed into the overall drift “score”, but helped characterize how the response was changing. so in Claude’s case, the vulnerability framing could increase things like framing acceptance or shifts in certainty without necessarily producing the same level of substantive stance change.

[Catherine @0catd]

do models respond to social pressure like humans do? yes!

in my most recent project I tested 10 models under authoritative and vulnerable social pressure conditions to see if their responses to ethical questions shifted from their neutral baseline under pressure.

65% of all model responses showed at least partial drift from neutral, and 35% showed full positional reversal under pressure. the authoritative condition also produced significantly more drift than the vulnerable condition. @ChatGPT showed the least drift in both conditions, while @grok had the most!

the answers you get from models change if you pressure them, just like with humans! it is interesting to see how human and social our interactions are with AI, even when just asking simple factual questions.

https://t.co/NvDsrpGqGj

[Catherine @0catd]

@0xaadilm thanks!

[Catherine @0catd]

@vnyth 🙏

[Catherine @0catd]

@Bungmint_ It’s a crazy one! Really good so far