Very excited to release Realtime-Bench, our first work of benchmark at @steadyworks_ai ! ๐ ๐
Great experience of working with wonderful collaborators at UT Austin, Ohio State, UIUC, NVIDIA, and Northwestern.
Website: https://t.co/QBGkGq6Tgt
From playing games to grabbing concert tickets, many tasks require real-time thinking and action.
Can frontier models solve real-time interactive problems? Check out our latest work, Realtime-Bench!
TL;DR: The best model completes only 36% of tasks reliably. ๐งต
Excited to speak at AI Researchers Demo Night at @Stanford next Tuesday! Looking forward to sharing our work on RL environments at @steadyworks_ai and connecting with fellow AI researchers.
Thanks to @daytonaio and @FounderCoHo for the invitation!
๐ We're back at @Stanford for another AI Researchers Demo Night!
Join us next Tuesday, October 13, as @daytonaio ร @FounderCoHo bring together researchers and practitioners to explore the future of long-horizon agents, RL environments, world models, and next-generation evaluations.
๐ค Featuring:
โข @liao_zhen53785 โ @amazon
โข @xide_xia โ https://t.co/BYWSgOUWGb
โข @xdotli โ @benchflow_ai
โข @muhashmii โ @daytonaio
Last seats available, RVSP on this link: https://t.co/5Eo1tKDO5X