Playing a lot lately with training loco-manipulation, here is a blogpost I wrote to compare the different data types used for training.
If you're in robot learning and deciding where your next hours of data should come from, I hope this map helps. https://t.co/h7hakD2wsX
@amypretzel Engineering is about solving problems under constraints - which can be summarized as an RLVR task... research is more about decision making - where the solution is unknown or might not even exist
A legged robot learned to walk, trot and bounce from minimizing only torque times joint velocity. No gait library, no reference motion, no reward for electricity ( the authors couldn’t measure it CoRL 2021, Unitree A1).
This is kinda like pricing your car’s fuel by engine RPM alone, nothing metering what idles under the hood.
Someone has now measured the real thing, on a different Unitree robot’s arm. Actual torque work was 10% of the electrical power draw. Copper loss was 42%. Friction, another 43%. The robot standing still, doing nothing, already pulled 135 W.
The proxy that produced a real gait never saw the terms that run the battery, as robots will start moving into deployment real energy spend would be harder to ignore.
Paper: Fu, Kumar, Malik & Pathak, CoRL 2021, arXiv:2111.01674
Paper: Deniz, Vega, Parsons & Auat Cheein, arXiv:2606.15915
@heyyalexwang So you view RL reasoning training more as imitation learning, and not real RL with exploration? why couldn't RLVR be used to check every reasoning step?
As Anthropic and DeepMind rolling out their new models with a major progress in robotics people are saying that specialized model or LLM alternatives are doomed.
Scaling laws work - if you continue to give the transformer more rich and diverse data it will learn it's distribution, yes even in robotics.
This doesn't downsize the training and data Fable 5.2 and Gemini Robotics 2.5 used to train. And it doesn't mean that a slow demo of solving a task means it will translate to a reliable and fast execution that will survive deployment.
@maximumbaelz@leothecurious We mostly don't understand the brain yeah, reliable in judgement, not necessarily performance. We might not know how we do it, but we can imagine outcomes before acting and choosing the best aligned action, this is not yet solved in statistical models
I suspect VLAs could be scaled the same way LLMs do, considering recent frontier ones even explictly trained on robotics data.
Figure’s Helix 2.0 allegedluy chained 61 actions over 4 minutes in a real kitchen, as ractive sequencing with no explicit lookahead.
Question is if the context managing we do with agent harnesses is enough for physical environments, and whether the representation of the current context is even correct for the physical world.
@leothecurious Well, not-brain-like architectures might lead us to solve brain-like architectures. We never had tools to compress and progress in distribution of human knowledge like this before
@cgtwts This isn’t really language anymore, tokens are very sparse in these model and the modalities are very diverse - it was clearly trained on robotics and 3d.
Anthropic bet on everything could be code really paid off, but representation isn’t the only problem in AI