Carlo Burelli

Political philosopher (realism, democracy, AI) and main writer of The King's & Queen's Dilemma.

[Carlo Burelli @BurelliCarlo]

AI safety already reasons like a realist: arms races, loss of control. Then it bets on giving the machine good values.

Our new paper argues the older fix: treat superintelligence as a hegemon, and coordinate against it through fear, not trust. Forthcoming in @EJPTheory

[Carlo Burelli @BurelliCarlo]

So I subbed to a gym and thought some gamification would help with motivation. I had Opus 5.5 build me a small app overnight, with quests, achievements and rewards. You start as a recruit. Top rank is Principe.

[Carlo Burelli @BurelliCarlo]

@vineerpasam ChatGPT is way easier to use for non tech people, as you don’t need to choose the right modell.

[Carlo Burelli @BurelliCarlo]

@enzoreds not much, if you believe the report (fair if you don't). Apparently, it took an average of around 3 hours per problem with a prompt as basic as "solve problem X"

[Carlo Burelli @BurelliCarlo]

@dioscuri Eerie choice of words, if you read between Mentor's lines.

[Carlo Burelli @BurelliCarlo]

At a certain point, asking whether a machine can really think becomes about as relevant as asking whether a submarine can really swim. Strictly speaking, no. But does it even matter, if it dives deeper, longer and faster?

[Carlo Burelli @BurelliCarlo]

@enzoreds Is it more submissions overall, or do you think the good/slop ratio has actually worsened? My expectation would be higher volume but also higher quality, not more slop.

[Carlo Burelli @BurelliCarlo]

@ValerioCapraro Agree there's no single morality to align to. But maybe that's the wrong target anyway: a more realistic goal is avoiding the extremely bad outcomes virtually everybody agrees on, rather than chasing the ideal good philosophers have debated forever.

[Carlo Burelli @BurelliCarlo]

@clairlemon Partly because, everything else being equal, a more aligned machine is a more capable machine: it's better at giving us exactly what we want. A practical example is RLHF, which started as an alignment technique and turned into a capability one.

[Carlo Burelli @BurelliCarlo]

@productaizery @collapsologist "the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking... Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view" Turing, 1950

[Carlo Burelli @BurelliCarlo]

https://t.co/ku9jZSdom9

[Carlo Burelli @BurelliCarlo]

Cool review of our board game, The Queen's Dilemma.

Is it too dark? It's not just me being a gloomy realist: some players optimize for points and ignore the story. A dark story asks: do you care enough about fictional points to authorize a fictional cruelty?

[Carlo Burelli @BurelliCarlo]

@ben_j_todd One from political theory, which isn't on the list yet (EJPT, with Federico Formentini): the safety literature's diagnosis is already realist (power-seeking, arms races, loss of control), but its solutions look for moral fixes to a structural problem.

https://t.co/nwsCGuyHt0

[Carlo Burelli @BurelliCarlo]

Tested Opus 5.5 by having it rebuild my website. It picked up the design of the old one, which took me ages by hand, and refined it in a handful of prompts.

Best part: from now on, a new paper or talk is just a line to Claude.

Early look: https://t.co/Lztinlf8e7

[Carlo Burelli @BurelliCarlo]

@BrianLeiter Fair. Intelligence here can just mean working out the best means to one's ends and acting on them, which current AI agents already do, albeit rudimentarily. "Super" just means far better at this than we are. Vague, sure, but not unintelligible.

[Carlo Burelli @BurelliCarlo]

@JulianL093 @DKokotajlo Even with a lot of resources a band of squirrels cannot evaluate whether humans are a danger to them.

The intelligence gap will be similar if not wider at some point in the future. Allthough we are not sure when (whether in 2, 10 or a 100 years)

[Carlo Burelli @BurelliCarlo]

@DarioAmodei This is almost point by point the agenda we defend in our new EJPT paper. If we are right, states should see that AI is not like nuclear weapons, which let whoever gets there first rule the others. It's a common threat: everyone gets dominated by the AI

https://t.co/nwsCGuy9Ds

[Carlo Burelli @BurelliCarlo]

People keep asking for the specific mechanism by which an AI could kill us all. But "I can't picture how" isn't evidence it can't.

A squirrel can't imagine how we'd burn down its forest, yet there are many ways we can. If (when?) super-intelligence arrives, we're the squirrels.

[Carlo Burelli @BurelliCarlo]

@DrDuncanBell A more extreme George the Chemist: take the job you find abhorrent (making chem weapons) because otherwise a more zealous candidate gets it and does it faster. Williams thought that argument was corrosive precisely because there's always someone worse waiting

[Carlo Burelli @BurelliCarlo]

@phl43 They are already better persuaders than most humans, and there’s increasing automation in factories laboratory and the military.