Excited to share our latest ICML 2026 paper: Solipsistic Superintelligence is Unlikely to be Cooperative
We argue that AI systems trained as isolated optimizers struggle during deployment in this fundamentally multi-agent reality. Why? The world pushes back. Humans adapt. Institutions adapt. Other AIs adapt.
As increasingly capable AI systems are deployed, humans, institutions, and other AI systems adapt in response — i.e. the world pushes back.
So is capability still the central safety challenge for AI?
We think not. We believe the harder challenge is coexistence.
The current AI research paradigm treats the world as a stationary source of feedback, what we refer to as the solipsistic approach to AI design. This raises serious risks for coexistence.
In our new #ICML2026 paper, we argue that superintelligence — an extremely capable task solver, built through such a solipsistic approach — is unlikely to be cooperative. 🧵
@Ugo_alves Yes absolutely it can. I agree social media is a primary method of diffusion of behaviors like virtue signaling. We also have a social media environment that you can play with to simulate this in Concordia https://t.co/hwsWSWDwPe
Ever wonder why we drop $1,000s on a Chanel bag or queue for ages for a Labubu doll? Status signaling drives a lot of human behavior, but how certain things become potent status symbols has a remained a big puzzle in social science
In our new paper we:
1. Synthesize the literature on this topic and propose a generative model of status signaling through the theory of appropriateness: people imitate what "someone like us" is supposed to want, display, and value
2. Show that we can simulate the theory to demonstrate how status symbols emerge with LLM-agent societies in Concordia
@jordigraumo@WilCunningham Sasha Vezhnevets and @jzl86
More broadly, I’m excited about this as a computational bridge from micro-level cognition to macro-level culture and economies. Huge thanks to my great co-authors!
Read the paper here: https://t.co/y82fRx708l
Code here: https://t.co/fPJeFLj71x
We generalize the same mechanisms beyond luxury consumption and procedurally generate social signaling scenarios for political allegiance, altruism, and arbitrary conventions.
For example: with a social life, agents were more likely to choose a large public donation over a smaller anonymous one
Check out our latest paper!
LLM-based social simulation is promising — but generic persona prompting collapses toward the “average human,” failing to capture the rich distribution of real behavior.
In Persona Generators, we introduce:
• A two-stage pipeline:
1.Generate thin personas that maximize coverage over diversity axes (explicitly sampling off-mode traits)
2.Expand each into a thick descriptive persona in parallel for scale
• LLM-guided evolution of the generator itself (AlphaEvolve-style optimization of code + prompts) to hill-climb on diversity
🧬 New paper from my internship at @GoogleDeepMind
We introduce Persona Generators: functions that generate diverse synthetic populations for arbitrary contexts.
We use AlphaEvolve to optimize the generator code, hill-climbing on diversity metrics — not just likelihood — counteracting the mode-seeking behavior of LLM sampling for agent-based simulations.
🧵👇1/
Some exciting job opportunities are now opening up on my team studying the post-AGI world.
If you know someone who would be a great fit for this, please pass this along!
[1/9] Excited to share our new paper "A Pragmatic View of AI Personhood" published today. We feel this topic is timely, and rapidly growing in importance as AI becomes agentic, as AI agents integrate further into the economy, and as more and more users encounter AI.