Can AI agents keep up with humans on real-time interactive tasks?
On Realtime-Bench:
๐ฎ Games: humans 72%, best model 32%
๐ป Complex web tasks: humans 7%, best model 52%
Humans react faster. Models are better at working out complex rules.
https://t.co/YPvXGpjD0M
โฑ๏ธ Introducing Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
The best model, GPT-6-Astra, solved only 36% of tasks reliably.
Jev achieves sub-1s latency, but its performance remains at 8%.
๐ Read our full blog: https://t.co/G9dzd1moWh