Asa Cooper Stickland

Poking language models @ the UK AI Security Institute

[Asa Cooper Stickland @AsaCoopStick]

NEW PAPER: Could an LLM agent subtly sabotage your code?

We conducted a red-blue team game where the red team designed agents to sabotage, and the blue team designed monitors to catch the agent.

Three surprising results ahead 🧵🛳️

[AI Security Institute @AISecurityInst]

AI coding agents are increasingly writing production code - with tools, file access, and execution permissions.

That power accelerates development, but also introduces new security risks if agents act against user intent 🧵

[Asa Cooper Stickland @AsaCoopStick]

.@soylent please come back to the UK we need you 🙏

[Huel @huel]

@AsaCoopStick @jjspicer May we never reach utopia

[Asa Cooper Stickland @AsaCoopStick]

If I ever give you a bad answer to a cyber-adjacent question that's because a dumber guy swaps in for those

[Asa Cooper Stickland @AsaCoopStick]

@opheliamoding @funplings lol yeah I did a bit of googling which said as much and realised maybe my intuition was coming from the people I meet which will probably select from roughly the same demographics as vibecamp

[Asa Cooper Stickland @AsaCoopStick]

@jjspicer I don't think EAs are imagining people still drink huel in the glorious utopia

[Asa Cooper Stickland @AsaCoopStick]

@underthenettle Extremely good plan

[Asa Cooper Stickland @AsaCoopStick]

GO TO STRAWBERRY HILL HOUSE

[Asa Cooper Stickland @AsaCoopStick]

So no lab has hired Raj Chetty yet? Are the recruiters all on holiday or what?

[Asa Cooper Stickland @AsaCoopStick]

@politicalmath I don't get the reasoning here. Like it genuinely bounces off my brain, can you expand?

[Asa Cooper Stickland @AsaCoopStick]

@draaglom I actually would have hit you up for a tour etc but was a family trip lol

[Asa Cooper Stickland @AsaCoopStick]

@sienna_rothery I should make the strawberry hill house partiful...

[Asa Cooper Stickland @AsaCoopStick]

@WillowChem Gothic revival architecture, slightly insane vibes, weird cool shit in every room...

[Asa Cooper Stickland @AsaCoopStick]

Nice work on v hard problem

[Mikhail Terekhov @MiTerekhov]

Our Anthropic Fellows project is now public!

The labs are planning to hand off AI safety research to AIs, but can we trust these AIs? We explore a way to control them for "fuzzy" tasks like writing research proposals. This is a whole new direction in diffuse AI control!

[Asa Cooper Stickland @AsaCoopStick]

@caryatis > i don’t have a speech describing myself ready to go

why not? surely you get similar ish questions often enough?

[Asa Cooper Stickland @AsaCoopStick]

@yong_zhengxin 👀

[Asa Cooper Stickland @AsaCoopStick]

I feel somewhat worried about AI safety as a whole optimising for empirical work/solutions that work for current models.

I don't really care about e.g. decision theory, but I think general macrostrategy/what do we do with aligned or "almost-aligned" AGI is v underinvested in

[Asa Cooper Stickland @AsaCoopStick]

Hmm seems good https://t.co/utwTTQQOqg

[Iliad @Iliad_research]

We are Iliad, and we do research and fieldbuilding to develop AI safety as a theoretical and experimental science. Learn more about us below! 🧵

[Asa Cooper Stickland @AsaCoopStick]

Back in the day I was long empirics, but I think we've managed to successfully scale empirical/near-term AI safety really well, and not so much future-facing AI safety. Another factor is the early advocates of theory were just crazy lesswrongers (much love to u guys) which made working with them less attractive to like "average cracked engineer/PhD student" and imo just meant they had less useful ideas. Feels like a new crop of more mainstream futurists has not emerged though

[Asa Cooper Stickland @AsaCoopStick]

@yong_zhengxin (and I guess important to create maximally-subtly misaligned models to test out any e.g. scalable oversight or "automate ai safety research" schemes)

[Asa Cooper Stickland @AsaCoopStick]

@yong_zhengxin Not really future-facing, but this is my favourite neglected non-future-facing area!