🤖✨Excited to share our new work:
OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
What if a humanoid could understand intent from language, music/audio, human motion, or their combinations—and turn it into executable whole-body motion in real time? [🧵1/11]
Here’s a pretty weird and surprising result - retrieval-augmented generation works unreasonably well for robot learning – but only when parameterized using difference vectors!
We introduce Difference-Aware Retrieval Policies for Imitation Learning (DARP), a simple, semi-parametric RAG architecture for imitation learning that achieves gains of up to 200% over standard behavior cloning. No additional assumptions beyond BC, just a little architecture switch! The theory backing it up is pretty cool too and it works on real robots! :)
Play with our website to understand better: https://t.co/4Ruk5aipTk
🧵(1/7)
Repo: https://t.co/nCBqFgxZyY Docs: https://t.co/2me2VADxQQ DOI: https://t.co/s1XYc5x280
Open-source in the spirit IKFast was. Especially eager to hear from folks building demo-collection rigs.
What arm do you wish had analytical IK?
In the last couple of months, we have witnessed significant advances in Industry-scale World Models. Yet, for the broader community, the gap between reading about these models and deploying them remains disappointingly wide.
Today we're releasing Nano World Models: a minimalist, batteries-included repo for advancing world model science.
🧵 (1/9)
Punchline: distill world models from simulation to enable fast, stable real-world robot adaptation.
Simulation is nearly always wrong. But in Simulation Distillation, we ask a simple question:
How do we perform simulation pretraining such that real-world adaptation becomes trivially easy?
https://t.co/ORDaxU2gzs
Let's take a closer look (1/n)
Real-world RL is still too brittle and data-hungry for long-horizon, contact-rich tasks.
We introduce Simulation Distillation (SimDist), which turns large-scale simulated experience into reusable world-model priors for rapid real-world adaptation.
By combining online planning with dynamics adaptation, SimDist achieves high success rates on tasks requiring precision, force, and reactivity.
Play with our interactive visualization to see for yourself: https://t.co/qFGNySxdAl
(1/n)
1/ 🤲 LeRobot has made low-cost robot learning widely accessible — but most policies are still blind to contact.
Today we release LeFlexiTac: a tactile extension for the LeRobot platform using FlexiTac sensors. Make tactile robot learning as easy as possible.
Project page: https://t.co/6PY8oTmAjU
Code/docs: https://t.co/11HW0Zwrtb
Most capable generalist robotics models today are closed or at best, open weights. But robotics won’t reach its ChatGPT moment without real openness.
That GPT moment was built on years of open tools and datasets such as Python, PyTorch, ImageNet and more, that let researchers inspect, reproduce, and build.
Today, we’re introducing MolmoAct 2: a fully open-source action reasoning model for real-world robotics.
We rethought and reshaped everything!
🧵👇
A touch-aware humanoid manipulation policy that cleans the lab for you🧹🧪
Introducing Humanoid Touch Dream: a real-world system for dexterous, contact-rich humanoid loco-manipulation.
Our key idea is simple: the policy predicts future hand forces and tactile latents alongside actions, within a single-stage training framework.
https://t.co/Pt5pXA65wm
1/7
What's different between these two BC policies? It's the same architecture, training budget, and data collection setup — the only difference is the controller gains!
Controller gains are an understudied design parameter in robot learning. In our new work (w/ @BronarsToni*, @pulkitology), we show how they act as an inductive bias across BC, RL, and Sim2Real transfer, with real consequences on performance. Here's what we found 🧵
* Equal Contribution
📄arxiv: https://t.co/SMYgh7i8cA
🔗website: https://t.co/cLCd1FYCdJ
🤖How do we evaluate robots?
Fixed benchmarks.
Same tasks.
Predefined success.
‼️But this doesn’t tell us what robots actually understand.
➡️We built **RoboPlayground**.
✨TLDR: Evaluation should be a space anyone can define, not a fixed benchmark.✨
🧵1/n
#robotics#ai#robot
🤲Tactile sensing is powerful for robot manipulation, but hardware is still difficult to access, reproduce, and scale.
🎯That’s why we built FlexiTac: an open-source, low-cost, and scalable tactile sensing solution designed for real robotic systems.
• Project page: https://t.co/i3TJxfkHOD
We hope FlexiTac can help democratize tactile sensing for robotics research. (1/n)
We’re building UWLab, a shared ecosystem for training robot policies in simulation and transferring them to the real world, built on Isaac Lab.
This includes the full OmniReset codebase, along with tasks, algorithms, and deployment in one clean, modular stack: https://t.co/PLX1fzPiSU
We’re releasing OmniReset, a framework for training robot policies using large-scale RL and diverse resets for contact-rich, dexterous manipulation.
OmniReset pushes the frontier of robustness and dexterity, without any reward engineering or demonstrations.
Try the policies yourself in our interactive simulator! https://t.co/3hW3nYx2vD
(1/N 🧵)
Excited to share the project that has surprised me the most in the last year!
Large-scale RL in simulation, no demos and no reward engineering can solve dynamic, dexterous and contact rich tasks. The learned behaviors are reactive, forceful and use the environment for recovery in ways that are extremely challenging to bake in or teleoperate!
You can play with the policies yourself to see: https://t.co/TCc4hb2baV
And, the learned behavior transfers to real world robots from RGB camera inputs!
So what’s the trick - using simulator resets carefully! Let’s unpack (1/10)
We’re releasing OmniReset, a framework for training robot policies using large-scale RL and diverse resets for contact-rich, dexterous manipulation.
OmniReset pushes the frontier of robustness and dexterity, without any reward engineering or demonstrations.
Try the policies yourself in our interactive simulator! https://t.co/3hW3nYx2vD
(1/N 🧵)
There’s a discussion going on rn about two recent robotic reward models: TOPReward⛰️ and Robometer🌡️
Which one is better? It depends entirely on your objective!
Here is a deep dive into the conceptual differences, strengths, and weaknesses of both. 🧵👇
A reward model that works, zero-shot, across robots, tasks, and scenes?
Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.
Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!
🧵 (1/12)