Wanted to get into manipulation. Decided to first build a full stack that scales.
Plan: copy @GeneralistAI's gripper. Fun weekend project.
It was not.
So @t_k_233 and I documented every design choice and lesson along the way.
Meet Specialist: fully open-source hardware, data pipeline, training, and deployment code.
https://t.co/aoN5x0Jp6I
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.
Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling
1300+ hrs, 240 scenes, 2000+ tasks, and growing
Dataset design, findings, and ecosystem 🧵
Our recent findings on World Action Models (WAMs): the core advantage of WAMs is not test-time “imagination” of futures, but the training-time supervision from future video prediction.
We propose Fast-WAM, which makes inference simple, fast, and policy-centric.
1/ World models are getting popular in robotics 🤖✨
But there’s a big problem: most are slow and break physical consistency over long horizons.
2/ Today we’re releasing Interactive World Simulator:
An action-conditioned world model that supports stable long-horizon interaction.
3/ Key result:
✅ 10+ minutes of interactive prediction
✅ 15 FPS
✅ on a single RTX 4090🔥
4/ Why this matters: it unlocks two critical robotics applications:
🚀 Scalable data generation for policy training
🧪 Faithful policy evaluation
5/ You can play with our world model NOW at https://t.co/SBqVDzYn86. NO git clone, NO pip install, NO python. Just click and play!
NOTE ⚠️
ALL videos here are generated purely by our model in pixel space! They are **NOT** from a real camera
More details coming 👇 (1/9)
#Robotics #AI #MachineLearning #WorldModels #RobotLearning #ImitationLearning
Train Beyond Language. We bet on the visual world as the critical next step alongside and beyond language modeling. So, we studied building foundation models from scratch with vision.
We share our exploration: visual representations, data, world modeling, architecture, and scaling behavior! [1/9]
my first PI project: we added memory!
this is a step function capabilities unlock: 15 minute long multi-step tasks in novel environments, controlled by text prompting
having run many of the evals, I legit think this is the GPT 2 moment for robotics
Instead of asking a VLM to output progress, it reads the model’s internal belief directly from token logits. No in-context learning. No fine-tuning. No reward training. 📈
We introduce: TOPReward, a zero-shot reward modeling approach for robotics using token probabilities from pretrained video VLMs.
The simplest way of doing reward modelling for robotics!
Project: https://t.co/LFtd2jAwwc
🧵👇
We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop.
Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate.
Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution.
Our recipe is called "EgoScale":
- Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks.
- Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency.
- Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone.
The scalable path to robot dexterity was never more robots. It was always us.
Deep dives in thread:
SONIC is now open-source!
Generalist whole-body teleoperation for EVERYONE!
Our team has long been building comprehensive pipelines for whole-body control, kinematic planner, and teleoperation, and they will all be shared.
This will be a continuous update; inference code + model already there, training code and gr00t integration coming soon!
Code: https://t.co/7u3SBxzXU9
Docs: https://t.co/HpDLkTCSMF
Site: https://t.co/D3i4KlnLLr
If we train VLAs to respond to diverse multimodal prompts, then we can steer them better: [grasp the carrot]/[move to x,y,z]/[put the carrot on the plate]. With many levels of detail, powerful VLMs can step in and steer the model to success much more often!
More below 👇
We’ve published our first technical report on Sprout.
Our paper outlines the design choices behind a lightweight, compliant humanoid platform built to operate safely in human environments and the system architecture that makes experimentation possible.
https://t.co/qCJaYQSUZ2
New milestone: we trained a robot foundation model on a world model backbone, and enabled zero-shot, open-world prompting capability for new verbs, nouns, and environments. If the world model can "dream" the right future in pixels, then the robot can execute well in motors. We call it "DreamZero", our first World Action Model (WAM).
Our team had tons of fun at the lab typing anything we like into an open text prompt, and watch the robot perform tasks it was never trained on. An emergent capability we didn't quite expect. Obviously not GPT-3 reliable yet, but we are marching into the GPT-2 era.
Discoveries:
- Model and data recipe co-evolve. Compared to VLAs, WAMs learn best from diverse data, breaking away from the conventional wisdom that lots of repeated demos per task are the bread and butter. Diversity >> repetitions.
- X-embodiment is extremely hard. Pixels are the answer. Different robot morphologies traditionally have a hard time sharing knowledge well. But if we put video first, pixels become the universal bridge connecting different hardware - even videos of human first-person view.
DreamZero shows significant robot2robot and human2robot transfer. With only 55 trajectories on a *new*, unseen hardware (~30 min of teleop), it adapts so quickly and retains zero-shot prompting ability.
Yesterday I posted about the "Second Pre-training Paradigm": world models are the next-gen foundation of Physical AI, not language backbones.
Today, we are proving it works. And 2026 has just begun.
Paper: World Action Models are Zero-Shot Policies.
Read it now: (thread)
[1/6] Ego-centric World Models
We introduce EgoWM — a video world model that simulates EVE-1X humanoid interactions from a single ego-view image + full-body joint angle trajectories.
Moreover it effortlessly generalizes to extreme OOD domains, including paintings !