🚨 New paper!
For LLM simulation to help with market and policy design, it has to hold up after the rules change.
Most evaluations stay in distribution. We choose what the simulator sees from past experiments, freeze it, and test on new people under new rules/design. 1/3
Prompting LLMs to behave like people can improve simulations, but how should we design this? What info is needed, and how much?
With @Kehang_Zhu@locross@bakkermichiel, we build “behavioral profiles” that help LLMs simulate human behavior under distribution shifts. 1/8 🧵
Yesterday I gave a talk at the eMERGE seminar at Berkeley.
Afterwards the organizers handed me a handwritten thank-you card. It really reminded me how much I've missed in-person seminars: hard questions in the room, hallway chats, and moments like this.
Thank you, @UCBerkeley 🐻
Is it just me, or are YouTube ads getting longer and increasingly unskippable?
The experience keeps getting worse. Even recordings of academic talks are interrupted by frequent ads now.
It’s really frustrating when you’re just trying to learn something.
Most tech breakthroughs happen once. Then everyone builds small improvements around them.
AI isn't like that. It's a continuous shock: it keeps getting better, and the progress is speeding up.
This slide from the Anthropic IPO roadshow was a big factor in my decision to leave. It’s an insane plot and I’m surprised they made it public.
It shows the compute efficiency of pretraining over time. The y-axis is a **log scale**: they are seeing continued superexponential algorithmic progress.
Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed.
It doesn’t strictly imply a software-only intelligence explosion- they have been spending more on R+D. But algorithmic improvements do not require ridiculous creative genius and are on the verge of being automated. You do the math.
The more I use AI agents, the more work they hand back to me for review.
I'm starting to feel like the bottleneck.
This fascinating Neuron paper estimates human information processing at just ~10 bits/s.
Maybe the ultimate bottleneck of AI agents is us.
2. More information isn’t always better. For incentivized behavior, a single well-chosen feature can be ~70% as predictive as a human. Adding more features can introduce noise rather than improve predictions.
To me, there are two big takeaways from this paper:
1. Incentives matter. To predict human behavior, we should study decisions with real consequences. Surveys and interviews often rely on “cheap talk,” which can introduce substantial bias. 1/2
Prompting LLMs to behave like people can improve simulations, but how should we design this? What info is needed, and how much?
With @Kehang_Zhu@locross@bakkermichiel, we build “behavioral profiles” that help LLMs simulate human behavior under distribution shifts. 1/8 🧵
🚨 New paper!
For LLM simulation to help with market and policy design, it has to hold up after the rules change.
Most evaluations stay in distribution. We choose what the simulator sees from past experiments, freeze it, and test on new people under new rules/design. 1/3
Prompting LLMs to behave like people can improve simulations, but how should we design this? What info is needed, and how much?
With @Kehang_Zhu@locross@bakkermichiel, we build “behavioral profiles” that help LLMs simulate human behavior under distribution shifts. 1/8 🧵
⚠️ The limit: profiles transfer within a game family, not reliably across families. Moving from one kind of game to another can do worse than no profile at all.
So re-validate before you cross. 3/3
One behavioral feature, chosen on earlier games, beat no profile and general-purpose personas in all 3 game families (public goods: 65.5 vs −1.0; 100 = a second human sample).
Surprisingly, more features didn’t help. In bargaining, the full profile fell from 77.5 to 53.1. 2/3
Recently rewatched The Matrix.
Neo realizes everything is just code. Once he can see it, he can rewrite it—bend the rules, reshape reality, even jailbreak an Agent.
Now LLMs are learning to read and edit neural network weights directly.
Are we getting closer to that moment?
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq
Time to update our priors on ABMs. Many still picture the Schelling’s model—dots on a grid with hand-crafted rules.
The Bank of Canada’s data-driven CANVAS model forecasts competitively with VARs and DSGEs.
A Lucas critique for ABM skeptics: models evolve; priors should too.
Data-driven economic ABMs are very interesting.
Poledna et al. (2023) is a compelling example: a national-scale economic simulation that rivals traditional VAR and DSGE models in out-of-sample forecasting.
https://t.co/vjgdf8XY8l
Exciting potential for ABMs × AI!
This is true and I have always thought it was a big shame. Part of the reason (as Andrey notes in the replies) is that historically ABMs had not been taken to the data. But this is changing. Here is my colleague @RMaria_drc’s recent work bridging this gap:
https://t.co/VCNUucO1wq
ABMs x AI hold a lot of promise for economics, and it’s an active area of research for our team.