🤖 Can giving AI agents more information make outcomes worse? 🤔
In our recent study of ten frontier LLMs, we explore how information shapes agents negotiating and trading on our behalf. More information can help them find better trade opportunities yet reduce collective gains.
1/ AI agents are already trading and bargaining on our behalf. But how does the information we give them shape their behavior & collective welfare? Does transparency always improve outcomes, or can it reduce efficiency? We study this across ten frontier LLMs in strategic markets.
🎉 Excited to share our #NeurIPS2026 paper!
🧠 Do LLMs have a reward system?
LLM hidden states are known to encode reward signals like correctness and confidence. But prior work fits black-box probes on the full hidden state, which says little about how this information is organized.
We find reward signals live in a small set of neurons:
🔹 Value neurons → predict state value
🔹 Dopamine neurons → encode step-level TD errors
🤖 Can giving AI agents more information make outcomes worse? 🤔
In our recent study of ten frontier LLMs, we explore how information shapes agents negotiating and trading on our behalf. More information can help them find better trade opportunities yet reduce collective gains.
1/ AI agents are already trading and bargaining on our behalf. But how does the information we give them shape their behavior & collective welfare? Does transparency always improve outcomes, or can it reduce efficiency? We study this across ten frontier LLMs in strategic markets.
1/ AI agents are already trading and bargaining on our behalf. But how does the information we give them shape their behavior & collective welfare? Does transparency always improve outcomes, or can it reduce efficiency? We study this across ten frontier LLMs in strategic markets.
🚀 Can AI scientists come up with truly creative ideas?
A creative idea needs more than good performance. It must be original, useful and non-obvious.
Yet AI scientists are usually given as much task information as possible, which anchors them on familiar approaches.
We introduce BVSR: Blind Variation, Selective Retention, a discovery framework inspired by a classic theory of human creativity.
✅ Plugs into existing AI-scientist workflows.
New Stanford paper shows that more information helps some LLM agents trade better and hurts others, depending on the model.
AI agents buying and selling in an auction were told exactly how much each side valued the goods, Claude Opus 5.5’s share of possible gains fell from 94% to 64%.
Multi-agent systems need rules on what each agent sees about others, such as exact prices or only rankings. The usual assumption is that more information helps.
They ran 10 frontier LLMs in trading, bargaining, and double auctions, changing only how precisely private values were shared, and scored gains against known utilities.
In trade, exact values cut GPT-6 Sol's gains from 98% to 91% of the maximum but raised Claude Sonnet 5's from 72% to 86%. In auctions, full disclosure inflated price demands and dropped Grok 4.7 from 97% to 62%.
When agents negotiate for you, test the model and what it gets told together, because more detail makes some models greedier.
Our new work on “Physics of Agents” https://t.co/qKo3O77kZ9 lead by Batu El and Jinhee Paeng in collab w/ @james_y_zou
The outcome of many interacting agents seems hard to reason about. Yet we were able to study the opinion dynamics of 10,000 different LLM agent communities as they communicated with each other to solve both objective and subjective questions.
Remarkably, we could account for their opinion dynamics through a simple Ising model that involved minimizing an energy function corresponding to social pressure to conform. Our dynamics could explain the build up of consensus, polarization, and societal correction of initially incorrect majorities.
Lots more to do on statistical mechanics of interacting agent dynamics!
New work I'm very excited about: Physics of Agents!🤖
As AI agents become more prevalent, they'll interact and influence one another.
Can we predict the collective behavior that emerges?
Surprisingly, their dynamics follow compact, predictive laws of statistical physics🧵
1/ Wonderful student projects from CS329H (Fall ’25) ML from Human Preferences at Stanford University! 🚀
@sangttruong@andreas_h0wpt and I introduced students to preference learning + alignment, culminating in final projects. Out of ~50, here are 5 standouts 👇
For anyone interested in understanding orthogonalization/spectral based methods here’s the slides from our #neurips25 oral that I tried to make more broadly about the topic.
Check out @Meterdogan27 's poster at NeurIPS. He just started his PhD at Stanford with one oral, one spotlight, and one poster presentation. Go talk to him at #909 today!
We are presenting our work on backprop alternatives, "Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism," co-authored with @Meterdogan27 & @CPehlevan, today at #Neurips2025!
📍 Poster S. 1, Exhibit Hall C,D,E - #2006 (Wed, 11am–2pm PST)
@CPehlevan@Alper_T_E If you’ll be at @NeurIPSConf, come say hi! We’d love to dive into the details of the paper, answer questions, or chat about any related ideas.
📍 Poster Session 1
🗓️ Wed Dec 3, 11am–2pm PST
📌 Exhibit Hall C,D,E — #2006
See you in San Diego! 🌴✈️
Excited to share that our paper “Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism”—with @CPehlevan and @Alper_T_E—was accepted as a #NeurIPS2025 Spotlight! 🎉
Paper:🔗 https://t.co/jtYQaZGXld
@CPehlevan@Alper_T_E We see this as a step toward unifying theoretically grounded objectives, biologically inspired credit assignment, and more flexible, hardware-efficient architectures, all without requiring symmetric backward weights.
@CPehlevan@Alper_T_E The resulting gradients naturally take a three-factor form:
• pre-synaptic activity
• post-synaptic nonlinearity
• a broadcast error signal
This structure mirrors the three-factor learning rules observed in biological neural circuits.
@CPehlevan@Alper_T_E We build a theoretical foundation for broadcast learning by leveraging a core MMSE principle: at optimum, the prediction error is orthogonal to any nonlinear function of the input. We enforce this via layerwise objectives that decorrelate the broadcast error from the activations.
@CPehlevan@Alper_T_E Backprop enables neural network training, but relies on a precise backward pathway that mirrors forward weights—a constraint that’s rigid for hardware and unlikely in biological systems. Error-broadcast methods bypass this by sending the output error directly to hidden layers.
@caglarml and I are excited to share our lecture slides for EE-628 Training Large Language Models course: https://t.co/QGRbwg9MKL
If you have any feedback, please reach out to us. I am also at #ICLR25.
Introducing the first NeRF-based 3D style transfer method that generalizes across scenes and styles!
Our approach eliminates the time-consuming optimization for each scene or style, making it more efficient than previous NeRF-based methods. Check it out! #GCPR2024