🤖 Robots are getting better at manipulation. But how far are we from robot assistants that can operate autonomously in the open world?
We introduce RobotAgentBench (RAB), a benchmark for open-world robot autonomy in realistic household environments.
We evaluate several VLA/WAM models and diverse robot agent harnesses across five capabilities: task discovery, context reasoning, manipulation, failure recovery, and robustness. Four open-ended household scenarios test how these capabilities work together to fulfill high-level user needs.
🔍 What did we find?
• Manipulation ≠ autonomy. Agent harnesses improve performance, but reliable open-world autonomy remains challenging.
• Hybrid execution works best in our evaluation. Combining learned policies with code-based control achieves the strongest overall performance.
• Robot agents improve through recursive self-improvement. By refining reusable tools and accumulating experience, agents improve across iterations.
We’re excited to share this work and look forward to the community’s feedback!
🌐 Project, code & data: https://t.co/KVDXyrNfJs
📄 Paper: https://t.co/kFdRukZGsW
Excited to finally share CRIS-0, Aether AI’s first causal robotic intelligence system! 🤖
It’s been an incredible journey building this alongside such a talented team at Aether AI. Grateful to everyone who made this possible.
Still a long way to go, and excited for what’s ahead
This is CRIS-0, Aether AI's first causal robotic intelligence system, operating in a real home. It reasons about how its actions will change the world, checks what actually happened, and adapts when reality doesn't go as expected.