cool demos, but not sure i completely understand the hype. it's hard to see these models ever deployed to manage the full control loop (inference/latency, would need compute that's orders of magnitude more efficient)
if the argument is that they'll be used for spatial reasoning / high-level planners that dispatch to local policies, we've already seen great models for this from gemini robotics (ER)
Fable 5.2 shows substantially improved skills in controlling physical robots, even on tasks it has never explicitly been trained to do.
On RoboDojo, a benchmark with a simulated two-arm robot, it outperformed other models and most AI systems built to control robots.
one interesting use case i can think of in deployments is as more of a last-resort interventionist. if a local policy fails, a frontier model could take over and recover the robot back into a state within distribution
Introducing CandyBench: a new frontier eval in applied robotics
we got 7 robots to give out 3057 halloween candies and graded them on performance
here's what happened:
Why create robot intelligence for just one hand, when we could have it learn from many?
GEN-1, our latest embodied foundation model, now supports a broad range of end effectors from 5-finger hands, to specialized tools, and everything in between.