Foundations of Cooperative AI Lab (FOCAL) at CMU

Foundations of Cooperative AI Lab @CarnegieMellon. Creating foundations of game theory for advanced AI w/ focus on achieving cooperation. Directed by @conitzer.

[Foundations of Cooperative AI Lab (FOCAL) at CMU @FOCAL_lab]

New positions at FOCAL! https://t.co/Wb1c7KqTiL

[Vincent Conitzer @conitzer]

We are recruiting postdocs at the Foundations of Cooperative AI Lab (@FOCAL_lab) at @CarnegieMellon (https://t.co/A72rHG8P87)! Please retweet / share / send great applicants our way! For different positions please reach out. @SCSatCMU @CSDatCMU @mldcmu
https://t.co/cY2yIeDXRb

[Vincent Conitzer @conitzer]

Two honorable mentions for papers at the ICML AI4GOOD workshop!
Paper led by @EmanuelTewolde and Xiao Zhang: ‘CoopEval’
https://t.co/wSf2JbyHOd
Paper led by @Akash190104 and @EmanuelTewolde: ‘Do LLMs Take Care of Their Own?’
https://t.co/iFXKQWPekN

[Vincent Conitzer @conitzer]

The AI4Good workshop is today (Korea) @ ICML! @EmanuelTewolde is presenting the CoopEval paper, but also follow-up work with CAIRF fellow Akash Kundu about whether LLM agents cooperate with others that they perceive as similar.
https://t.co/meZ8P3NLVw
https://t.co/kYDYxrAZXe

[Vincent Conitzer @conitzer]

Today (Korea time) at ICML in the 5pm session, Vijay Keswani is presenting our position paper "We Need Practical AI Alignment Methods that Mirror Human Reasoning!"
presentation: https://t.co/uWQDOBnOaQ
paper: https://t.co/ZdAmo1I2b8

[Emanuel Tewolde @EmanuelTewolde]

Standing by the CoopEval poster right now at #ICML2026! 📊👋

If you are interested in cooperation and multi-agent safety, game-theoretic mechanisms, or modern LLM agents defaulting to defection, come by board #4008 and let's chat! 🇰🇷🫱🏿‍🫲🏾

[Vincent Conitzer @conitzer]

@EmanuelTewolde is presenting our CoopEval work at ICML on Wednesday 10:30am session (or catch him at the alignment workshop today)!
presentation: https://t.co/It7BOxNhlg
arXiv: https://t.co/wSf2JbyHOd

[Vincent Conitzer @conitzer]

Congrats to our recent undergrad+MS grad Jerick Shi on receiving the best paper award at ICML NExT-Game for our paper "When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games" w/ Terry J. C. Zhang, Bernhard Schölkopf, @ZhijingJin!
https://t.co/hClDFQbabl

[Vincent Conitzer @conitzer]

our recursive joint simulation paper (w/ Vojta and @C_Oesterheld) accepted to Synthese! TLDR: When players in a game run a simulation of themselves (incl. further subsimulations), that's equivalent to an infinitely repeated game and so allows cooperation. https://t.co/uDpSfjnZHe

[Vincent Conitzer @conitzer]

One of my open math problems apparently got resolved by ChatGPT 5.5 Pro (Ryan O'Donnell prompted it better than I did!), though the proof was so hard for me to read that it seemed easier to just prove it myself. More thoughts on implications for math here: https://t.co/8NGs5sVzzn

[Vincent Conitzer @conitzer]

@FOCAL_lab @EmanuelTewolde https://t.co/wSf2JbyHOd

[Vincent Conitzer @conitzer]

@FOCAL_lab member @EmanuelTewolde is presenting his CoopEval work at EPFL on Monday (14:30) and it will be on zoom! (paper link in comment)
https://t.co/Lv3eoBFD8o

[Jiayuan Liu @jiayuan_liu_]

(1/4) Can remembering more of the past make AI agents less cooperative?

In our new paper, we study LLM agents in repeated social dilemmas. The key variable is not how many rounds they play, but how much prior interaction history they can access when making each decision.

[Jiayuan Liu @jiayuan_liu_]

(2/4) Surprisingly, longer recall often degrades cooperation.

Across 7 LLMs and 4 repeated social dilemma games, agents with longer histories often shift away from forward-looking cooperation and toward retrospective grievance-tracking.

[Jiayuan Liu @jiayuan_liu_]

(3/4) The mechanism is not just “too much context.” It is what the agents remember: replacing histories with synthetic cooperative records restores cooperation, and ablating explicit CoT reasoning often reduces the collapse.

We call this the memory curse.

[Jiayuan Liu @jiayuan_liu_]

(4/4) Paper link: https://t.co/UUPIzVzCNy

Big thanks to my collaborators! @Jack_Litq, @zzzoooeee321, Xin Luo, Haoxuan Zeng, @EmanuelTewolde, Tai Sing Lee, @tonghanwang, @ckingsford, @conitzer

[Vincent Conitzer @conitzer]

In this paper led by Jerick Shi @Jerick1380, we propose a taxonomy of LLM deception. https://t.co/VpYl2ld98I

[Vincent Conitzer @conitzer]

(3/3) Also, I don't understand how some people think AGI is just around the corner but the risks are easily manageable! Of course their positions may not be captured accurately here.

[Vincent Conitzer @conitzer]

(2/3) I'm sure we all have thoughts on our descriptions -- I certainly worry about many other AI risks current and future in addition to scaled misinformation, and I actually think the world is too focused on LLMs-as-chatbots -- but still impressive.

[Vincent Conitzer @conitzer]

(1/3) Wow, I (and many friends) made it onto this "Mapping AI" map! Does anyone know the methodology? Who is behind it?
https://t.co/QhqHW5ijDE

[Vincent Conitzer @conitzer]

(3/3) You can play with interleaving tokens yourself here! (... works better with better models...)
https://t.co/GUVwkbWd25

[Vincent Conitzer @conitzer]

(2/3) We've also been interested in interleaving tokens for philosophical reasons. This chapter based on a talk I gave at a Duke conference about tests of consciousness discusses how coherent LLM text doesn't necessarily come from any clear unit entity.
https://t.co/Yei0he6u64