Context isn’t just what you put in a prompt. It’s also what stands between a robot and its goal. Robot policies in everyday environments must generalize not only across objects, but also across the spatial contexts in which they appear.
Excited to share COWBOY 🤠 (CoRL 2026) — our sim-to-real recipe for contextual whole-body manipulation. One generalist grasping policy coordinates base, arm, dexterous hand, and active vision end-to-end, adapting its motion and grasps to object and scene geometry. Trained in simulation, deployed zero-shot in the wild.
w/ @JasonJZLiu@LiuMuxinl
🧵(1/N)
I’m glad that Skild emphasizes that the tasks they test on in this demo aren’t found anywhere in the pre-training set. That means everything for a foundation model.
I’m feeling incredibly lucky to be working with @deepakpathak in both academia and industry to solve robotics!
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning:
Got to visit the Robotics Institute at CMU today.
The institute has a long legacy of pioneering research and pushing the frontiers of robotics.
Thanks @kenny__shaw@JasonJZLiu@adamhkan4 for showing your latest projects. Here’s a live autonomous demo trained with DexWild data