Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!
@elonmusk Thinking in language has limited applications, largely in coding and mathematics where the language itself can help reasoning.
But, as I've been saying for years, thinking manipulates mental models in abstract (continuous) representation space.
Soooo, xAI gonna use JEPA now?
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
NEWS: Boston Dynamics has just released a new video of its upgraded next-generation humanoid robot called Atlas.
• 4 hour battery. Self-swappable for continuous operation
• 6 feet 2 inches tall
• Weight: 198 lbs
• 56 total degrees of freedom
• Now fully electric, ditching older hydraulic systems
• New lightweight mix of aluminum and titanium components
• 110 lbs weight capacity (66 lbs sustained)
• Can reach up to 7.5 ft
• Constantly evaluates its surroundings and adjusts its posture, balance, and grip in real time
• Hands that can reconfigure as needed. Tactile sensors feed data back into the system, helping apply the right amount of force
• Brain is powered by Nvidia chips
We got our robots to wash pans, clean windows, make peanut butter sandwiches, and more!
Fine-tuning our latest model enables all of these tasks, and this has interesting implications for robotics, Moravec's paradox, and the future of large models in embodied AI.
More below!
Robotics is NOT a data problem. Atleast not the way we think.
For a long time, my world model was: whoever deploys robots at scale first wins. Get the units out there, collect the trajectories, train on the data, repeat. The fleet learning flywheel.
I'm starting to think this is wrong.
Consider Tesla. They've been collecting driving data for nearly a decade. Millions of cars, countless edge cases, every country, every weather condition. If any company has the data flywheel spinning, it's them. And FSD is genuinely impressive now and works insanely good. And yet. We're still 2-3 years minimum from actual robotaxis without remote operators or telepresence. Maybe a decade from global deployment. This is with ten years of data collection on what is, in some sense, a constrained problem. Roads have lanes, cars have physics, the state space is bounded.
Now take a similar analogy for general-purpose robotics. Homes are weird. Objects are weird. Tasks are open-ended. If the driving problem took this long with this much data, why would we expect manipulation to be faster?
The standard counterargument goes something like: "But we have foundation models now! VLMs! World models! All that semantic and physics understanding is already baked in, so the timeline to deploy general robots should be much smaller than what it took Tesla."
I used to believe this. I don't anymore.
Here's what I think is actually happening: current world models understand physics the way someone who's only ever watched videos understands physics. They know things fall. They don't know how fast or where exactly. They have vibes about spatial relationships but no ground truth. This matters more than people realize.
You see the same thing in software, actually. No matter how good Claude Code gets, it still sometimes gets stuck in a loop; fixing B and destroying A, then putting A back and destroying B. It can write impressive code, but it doesn't have a closed-loop understanding of what it's actually doing to the system. It's the same problem. Vibes about code, no ground truth about state.
I've been messing around with some weirdly specific measurement problems lately. Figuring out how much a fish weighs from a photo, estimating cattle mass from iPhone LiDAR, that kind of thing. Very niche, very applied. But working on this stuff and my robotics hobby has updated me on something: getting actual ground truth about the physical world is much harder than I'd assumed, and much more valuable than the foundation model people seem to think.
There's something deeply wrong with training a "world model" on pixels alone and expecting it to understand geometry. It's like training a language model on text and expecting it to understand truth. You get something that's eerily convincing and subtly broken.
The current robotics paradigm is: go to 1000 homes, teleoperate 1000 robots, collect the data, train on it, pray for emergence. This gives the vibes of "we're losing money on every sale but we'll make it up in volume." If your learning algorithm requires 10,000 robots doing teleoperated data collection before it starts being useful, you don't have a learning algorithm, you have a very expensive imitation pipeline.
I do want to believe we have everything we need to build general-purpose robots. Deep learning and transformers have surprised us in shocking ways. The kind of emergence that led to ChatGPT was incredible. But at the same time, you still cannot deploy a model as a co-worker. And I think it's the same with robotics.
The more I learn about this tech and actually try to deploy it, the more my intuition says there's something missing. Once we figure that out, things will fall into place much more orderly than the current approach of going to 1000 homes and deploying 1000 robots to collect data and then expecting emergent skills.
It shouldn't be that way. We should be able to figure out how to provide value from the beginning.
I don't know exactly what that missing piece looks like. But I'm pretty sure it's not "more demonstrations."
VLAs and robots have real use cases today. It's the big G that I'm uncertain about. And as always, would love to be wrong.