Modern businesses run on spreadsheets. Can AI handle financial spreadsheet construction?
We’re excited to share MBABench 2.0: 101 difficult, long-horizon tasks (take up to 33h) across 6 finance areas and most financial model types.
The best AI agent passes only 16% of the tasks
Introducing GameBoyWorlds, a new challenge testbed for self-improvement in embodied video games:
Paper: https://t.co/05Fpom1OM1
Website: https://t.co/qFUZfld80b
Today we're releasing MBABench 1.0: the first, spreadsheet-focused benchmark evaluating AI agents on building entire workbooks to professional standards.
In the real-world, MBAs must produce complete, well-organized artifacts ranging from spreadsheets to presentations. These deliverables are then reviewed and revised by multiple stakeholders ranging from managing directors to client analysts, so beyond accuracy, they must meet various quality standards specific to the industry.
We evaluated 10+ frontier models on end-to-end spreadsheet tasks, and found agents still fall short of the standards MBA grads are held to. (1/n)
I'm excited to release MBABench, a new eval for nuanced judgement-driven tasks that MBAs perform. Being able to interact with professionals in NYC has provided many unexpected learnings on the impact of AI in white-collar work, and this project reflects this rich confluence!
Today we're releasing MBABench 1.0: the first, spreadsheet-focused benchmark evaluating AI agents on building entire workbooks to professional standards.
In the real-world, MBAs must produce complete, well-organized artifacts ranging from spreadsheets to presentations. These deliverables are then reviewed and revised by multiple stakeholders ranging from managing directors to client analysts, so beyond accuracy, they must meet various quality standards specific to the industry.
We evaluated 10+ frontier models on end-to-end spreadsheet tasks, and found agents still fall short of the standards MBA grads are held to. (1/n)
Alibaba quietly open-sourced LOGOS: a 1B foundation model, 56x smaller than Microsoft's NatureLM and still beats it across proteins, molecules, and materials.
https://t.co/ncQpMwOq0A
ByteDance just published something I've been waiting for someone to build: CUDA Agent!
It trained a model that writes fast CUDA kernels. Not just correct ones — actually optimized ones.
It beats torch.compile by 2× on simple/medium kernels, ~92% on complex ones, and even outperforms Claude Opus 4.5 and Gemini 3 Pro by ~40% on the hardest setting.
The key idea is simple but kind of brilliant:
CUDA performance isn’t about correctness, it’s about hardware. Warps, memory bandwidth, bank conflicts — the stuff you only see in a profiler.
So instead of rewarding “did it compile?”, they reward actual GPU speed. Real profiling numbers. RL trained directly on performance.
That’s a big shift.
Paper: https://t.co/EYx7QKosgk
Project: https://t.co/pTCfzQIBes
@odd_joel I instantly get you a sandbox with persistent storage so that you don't have to worry about your laptop being on all the time + don't need to set your own VM + you will get preview links. Custom Websockets for connection. Voice-to-terminal sounds cool!