My biggest struggle with reviews in 2026 is that (AI) reviewers donβt seem to know the difference between limitations and critical issues. An LLM can easily find 100+ limitations in any paper (even the best ones in history), but that doesnβt mean the paper shouldnβt be accepted.
β±οΈ Introducing Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
The best model, GPT-6-Astra, solved only 36% of tasks reliably.
Jev achieves sub-1s latency, but its performance remains at 8%.
π Read our full blog: https://t.co/G9dzd1moWh
From playing games to grabbing concert tickets, many tasks require real-time thinking and action.
Can frontier models solve real-time interactive problems? Check out our latest work, Realtime-Bench!
TL;DR: The best model completes only 36% of tasks reliably. π§΅
the rich can brute-force their way through hard problems (10,000 agents and a few million $ in API). The poor have to keep optimizing their search algorithms π
System prompts decide what an AI tells you, hides from you, and nudges you toward. And you rarely get to read them. They indexed 1,000+ prompts from 400+ products, audited 88 in depth: ~40% contain at least one instruction working against the user's interests.
@joabaum@icmlconf Congrats, well deserved! π We found a similar phenomenon: gaming AI reviewers with presentation-only edits, science fully fixed. Additionally, strengths amplify more easily than weaknesses dissolve, and reframing how reviewers read it beats polishing. Hope to see you at ICML β
Grateful to all collaborators on this one π Thanks for reading!
Authors: Xu Yang, Zhizhou Sha @ZhizhouSha, Junbo Li @ljb121002, Jian Yu, Yifan Sun, Matthew Zhao, Jinrui Fang, Xinyue Guo, Yining Wu, Xu Hu, Yifu Luo, Qiang Liu, Zhangyang Wang @VITAGroupUT
Affiliation: @UTAustin
π§΅ AI reviewers are rolling into peer review. Can you game them with no hidden text or prompt injection at all?
Yes, by presentation-only edits, with no changes to methods, figures, equations, or numerical results.
β 75.1% attack success, +1.21/10 mean score gain π
Why it matters: this isn't one model's quirk. It holds across reviewer models and venue templates, structural, not a single-model bug.
Right now, AI reviewers fail this: presentation alone moves the verdict. At scale, it risks shifting authors from better science to packaging.