Runtime

Evaluation Infrastructure for Robotics (YC F26)

[Runtime @runtimesi]

Introducing SimInfinity -- the world's largest open-source robotic simulation benchmark
Coming soon: 10K open-source simulations with well-defined robotic tasks inspired by real-world activities across factories, logistics, homes, and beyond.

[Runtime @runtimesi]

Check out the full writeup here: https://t.co/iW7dt6Zxxk

[Runtime @runtimesi]

Verbal reinforcement learning is very effective for improving agentic performance on benchmarks, but we wanted to test how this transfers to robotics. We ran GPT 6 Astra, Gemini Argon 4, and Fable 5.1 on the LIBERO benchmark, and then iteratively developed skills which significantly improved performance.
Full results linked in the comments

[Runtime @runtimesi]

Frontier models like GPT-6 Astra and Gemini Argon 4 can control robots, but how much does the harness around them impact performance?
We used our Nomad-Harness to test Gemini Argon on LIBERO tasks to test if text based skills could help with harder tasks that require more manipulation. We iteratively refined the skill with a critic agent reviewing failed attempts.

Across 20 initial states per task, Argon’s native success rate improved:
• Bowl into drawer: 0/20 → 17/20
• Mug into microwave: 0/20 → 8/20

These results show that reusable text instructions can improve robot control, and they also transfer between models, with improvements also shown for GPT-6-Astra and Fable 5.1 with the same skill.

[Runtime @runtimesi]

🚀 Runtime (YC F26): The simulation and evaluation platform for physical AI Build realistic environments, stress-test robot policies across thousands of variations, and identify failures before real-world deployment. Building robots? We'd love to chat!

[Runtime @runtimesi]

🚀 Runtime (YC F26): The simulation and evaluation platform for physical AI

Build realistic environments, stress-test robot policies across thousands of variations, and identify failures before real-world deployment.

Building robots? We'd love to chat! Reach out to us at https://t.co/AAZNT7qaUO