Arthur Conmy

@anthropicai

prev fixing things @googledeepmind

[Arthur Conmy @ArthurConmy]

I'm joining Anthropic!

I'll start work on aligning upcoming models as they’re trained

Claude's capabilities are extraordinary. But like all models thus far, Claude isn’t aligned enough to safely delegate AGI development to

I can't think of a better place to work on this at

[Arthur Conmy @ArthurConmy]

@_achan96_ See https://t.co/mdZLwqRxex

[Arthur Conmy @ArthurConmy]

@eterecursion @uzpg_ Actually it's not that bad just figure assumed all caps lora already trained

[Arthur Conmy @ArthurConmy]

@eterecursion @uzpg_ It's quite hard to read this https://t.co/vVxufWXdKo given fable chose a different example of traits than the img

[Samuel Ratnam @eterecursion]

The method: train the base model underneath a frozen LoRA.

Train a small LoRA that elicits trigger trait X (e.g. all-caps speech)
Freeze it. With adapter ON, train the base on payload-trait-Y responses (owl love). Adapter OFF, train on clean responses
Remove the adapter

[Arthur Conmy @ArthurConmy]

@eterecursion @thkostolansky but also competence, agency, truth-seeking, persistence,

[Arthur Conmy @ArthurConmy]

@Tim_Hua_ Plausibly related to https://t.co/sj14CM3ikI (self-esteem is surely correlated)

[Arthur Conmy @ArthurConmy]

you have push access to https://arthurconmy.githu
https://t.co/vnoSG2e5Vt
it-teacher__explicit-america__bm-1 The
repository where this is in. I want you to make
me a flash game from scratch. Some of the flash
games I enjoy when I are younger are like
Fireboy and Watergirl as well as Commando 2 was
pretty good and <REDACTED NAME>, you listen to...
Sportshead Tennis. Football. Sportshead Tennis.
We like that one. Sportshead Football. Yeah.
Sportshead Football, that was good as well.
What's the little prisoner one we're playing?
Some prisoner game relates to these ones and
just like it must be new. Must not be a copy of
an old one whatsoever. Think of a new idea.
Supervitas was a good one as well. Make that
from scratch, push it to my website so when I
load my website on this Mac then I can play that
game. Thanks, go.

[Arthur Conmy @ArthurConmy]

Had a 'feel the AGI' moment: got Fable to make a 2-player flash game for friends outside tech and it actually created a good vibe!

[Arthur Conmy @ArthurConmy]

@mentalgeorge SuperWhisper with many voices after the first bit lol

[Arthur Conmy @ArthurConmy]

-They didn't really believe this wasn't a copy of an existing game (memorization/plagiarism). This drives a lot of negative sentiment about AI at least in the UK

-Past that they were quite impressed, in particular curious and surprised about how Claude Code can do so much more than web chat bots theyd used

[Arthur Conmy @ArthurConmy]

I haven't tried the baseline with other models, likely this isn't Fable specific. But one shot, one superwhisper prompt to minutes later having something of (small) value that has never been made before, comprehensible to those without context is insane!

[Arthur Conmy @ArthurConmy]

https://t.co/YDkaTpXStX

[Arthur Conmy @ArthurConmy]

"forward model" and "lexemes" and "continuation top-20" and "clause-terminal" read as really non-standard phrases to me

[Arthur Conmy @ArthurConmy]

It seems to me that AI writing has changed, at least on technical topics. One year ago every AI was far too verbose always. But recent Opus outputs are sometimes not verbose enough. "Stacatto" for sure. Reminds me of this "split personality" point from @nostalgebraist

[Kevin Patrick Murphy @sirbayes]

While doing my NeurIPS reviews, my "claude generated paper" trigger kept going off - there is a distinctive style that they have - very stacatto, terse, defensive. What happens when LLMs start training on this text? Is the fixed point even readable to humans?!

[Arthur Conmy @ArthurConmy]

@Butanium_ It is genuinely hard. https://t.co/QTGMLm1kGN

and the prompt says do not cheat. In this MirrorCode case it's a narrower benchmark so was made cheat proof. Im not sure how scalable that is to more general benchmarks

[Arthur Conmy @ArthurConmy]

😍 A benchmark that actually prevents hacking!

[Epoch AI @EpochAIResearch]

MirrorCode resists cheating by design. We sandbox AIs: no internet access, no way to get the original source code, no hacking the scorer. Models never see held-out tests while developing their code, so they cannot cheat by creating a lookup table against the original program.

[Arthur Conmy @ArthurConmy]

@d_yuqing Thanks Yuqing! And I enjoyed working with you too!

[Arthur Conmy @ArthurConmy]

I'm joining Anthropic!

I'll start work on aligning upcoming models as they’re trained

Claude's capabilities are extraordinary. But like all models thus far, Claude isn’t aligned enough to safely delegate AGI development to

I can't think of a better place to work on this at

[Arthur Conmy @ArthurConmy]

@NeelNanda5 Thanks Neel for everything! 🥲

It is sad they we will not be able to collaborate as closely, but I took so much from your management and mentoring over the last 3 years

[Arthur Conmy @ArthurConmy]

@Roprajo thanks Pratik! Likewise, sad it was brief!

[Arthur Conmy @ArthurConmy]

I’ll move from London to SF for this work! 🥲

[Arthur Conmy @ArthurConmy]

Q: What do I mean by aligning upcoming models?

A: Triaging signs of misalignment in training, then aiming for root-cause fixes over whack-a-mole patches (e.g. training against the behavior). This post is the best work I’ve seen on how to align models: https://t.co/NavWn0lKc6