What excites me about COWBOY is how it scales: train local manipulation skills entirely in sim, then combine them into one whole-body policy. Zero teleop or human demos 🤠
Honored to be a co-first author—learned a ton, and grateful to the team!
@Jiahui_Yang6709@JasonJZLiu
Context isn’t just what you put in a prompt. It’s also what stands between a robot and its goal. Robot policies in everyday environments must generalize not only across objects, but also across the spatial contexts in which they appear.
Excited to share COWBOY 🤠 (CoRL 2026) — our sim-to-real recipe for contextual whole-body manipulation. One generalist grasping policy coordinates base, arm, dexterous hand, and active vision end-to-end, adapting its motion and grasps to object and scene geometry. Trained in simulation, deployed zero-shot in the wild.
w/ @JasonJZLiu@LiuMuxinl
🧵(1/N)
Today we’re open-sourcing Steel, our internal physics solver built with @NVIDIA.
It runs simulations up to 13× faster than PhysX while using ~3× less GPU memory.
Steel unlocks scaleable simulation for manipulation, for everyone.
Excited to finally share Pixel: a full-stack robot learning project @_tonytao_ and I worked on earlier this year!
We’re releasing two papers:
1. Pixel: A low-cost mobile manipulator, co-designed with its UMI system and learning stack. https://t.co/GZciGuUaBp
2. Pixel-UMI: Improved UMI pipeline for learning environment-aware, whole-body manipulation
https://t.co/dG054obq6x
(1/N)
Back in ~Dec 2025, I really wanted to work on mobile manipulation, but robots were either too expensive or not performant enough.
So we built our own!
Excited to finally share something I’ve had so much fun working on:
1. Pixel: a co-designed full stack system for learning mobile manipulation.
2. Pixel-UMI: environment aware whole body manipulation without robot data.
w/ @JasonJZLiu
(1/N)
Excited to finally share Pixel: a full-stack robot learning project @_tonytao_ and I worked on earlier this year!
We’re releasing two papers:
1. Pixel: A low-cost mobile manipulator, co-designed with its UMI system and learning stack. https://t.co/GZciGuUaBp
2. Pixel-UMI: Improved UMI pipeline for learning environment-aware, whole-body manipulation
https://t.co/dG054obq6x
(1/N)
The results are incredibly impressive, but I’m even more excited by the direction. Huge respect for a startup willing to invest in fundamental questions that may matter for robotics in the long run, rather than just chasing the fastest short-term wins.
The only AGI on Earth emerged after billions of years of "physical self-play". Robots won't be any different.
This method scales, and we will scale it.
The only AGI on Earth emerged after billions of years of "physical self-play". Robots won't be any different.
This method scales, and we will scale it.
@deepakpathak It also computes confidence scores, which seems to be great for selecting the better action if we let the VLA compute a batch of actions. Is this a reliable approach?
Probably the strongest robot foundation model I’ve seen so far.
One model adapting to new tasks from context, without task-specific fine-tuning. Pretty amazing that this is possible now.
So cool to see @deepakpathak and the @SkildAI team push this forward 🚀
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning:
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning:
We’ve released code for NEXT, our neural external torque estimation method from FACTR 2!
NEXT enables commodity robot arms to estimate external joint torques without dedicated force sensors. We hope this release will be useful to the robotics community.
Code: https://t.co/i9CdZmpn1W
Website: https://t.co/2TFoYc0hBC
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors.
We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies.
FACTR 2 consists of:
1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors.
2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training.
w/ @StevenOh_@_tonytao_
🧵(1/N)
@Matagi1996@Etienne547@cloneofsimo DINO is not JEPA. It might appear to be similar to JEPA. Reading the two papers carefully tells you the difference. Also, I don’t think there’s a self-supervised model that can beat SOTA methods on imagenet with only linear probing and a frozen backbone. Even DINOv3 has <90%.
Human grasp data is scarce. How do we teach robot hands to grasp like us?
Meet IFG, our framework for generating internet-scale, semantic-aware, dexterous grasp data without any manual human data collection.
https://t.co/o6YOnrsZOd
@Etienne547@cloneofsimo Yeah, it’s catching up to SOTA models like DINO. Self-supervised models don’t have class labels during pretraining, so it’s normal for them not to be as good as supervised ones.
Ever wish a robot could just move to any goal in any environment—avoiding all collisions and reacting in real time?
🚀Excited to share our #CoRL2025 paper, Deep Reactive Policy (DRP), a learning-based motion planner that navigates complex scenes with moving obstacles—directly from point cloud input.
w/ @Jiahui_Yang6709
(1/N)