AI is using more and more of the world’s energy. But how much economic value are we getting from AI?
We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.
Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:
1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.
2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.
3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.
With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.
Joint work co-led by @amirzeinaalii and @Avanika15, with @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, @HazyResearch, and @Azaliamirh.
More details below 🧵
Can an LM, starting from random init (!!), learn to generate all of its pretraining data?
Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities.
A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya@noahdgoodman, and @YoavLevine.
You're wasting FLOPs when scaling inference compute: by independently sampling parallel attempts, you burn compute rediscovering the same solutions.
Introducing QuasiMoTTo: we scale parallel sampling with correlated samples instead! These samples have higher coverage, are marginally exact draws from the LLM, and can be generated in parallel.
Result: same performance with 25-47% fewer samples in test-time scaling + 50% fewer training steps in RL!
In our new paper, we explore the design space of correlated samplers. Work with co-authors @probablynotaz9 (co-lead), @gandhikanishk, @noahdgoodman, and Emily Fox!
AlphaFold-like models such as Boltz-2 and AlphaFold3 learn powerful priors over structures from PDB. But how can we make them aware of new X-ray crystallography data?
Introducing CrystalBoltz: end-to-end X-ray structure determination via experiment-guided diffusion.
1/4 🧵
Just released the latest project from my PhD: a method for fine-tuning Boltz-2 to output atomic conformational ensembles by supervising directly against cryo-EM maps. Take a look at the thread to learn more!
AlphaFold-based models like Boltz-2 and BioEmu train on atomic conformational structures in order to predict protein dynamics. But is it possible to train these models directly on cryo-EM map ensembles, harnessing conformational data that is typically not deposited in the PDB?
Introducing CryoSampler: a new approach for fine-tuning Boltz-2 with raw supervision on cryo-EM map ensembles.
1/6🧵
Can a language model learn, end-to-end, what to keep in its own KV cache and what to throw away? Can it learn to forget while it learns to reason?
Deep learning's central lesson: capability emerges from end-to-end optimization, not heuristics/strong inductive biases. But for efficiency, we rely heavily on hand-designed approaches.
🗑️ Introducing Neural Garbage Collection (NGC): we train a language model to jointly reason and manage its own KV cache, using reinforcement learning with outcome-based task reward alone. No SFT, no proxy objectives, no summarization in natural language.
New paper with @jubayer_hamid, Emily Fox, and @noahdgoodman!