Your world model needs fun data rather than numerous repeated demos. Let robots explore and collect data that maximizes action influence, and it brings up to 85x data-scaling efficiency of expert data!
Glad to contribute to this interesting project 🤖
https://t.co/2IKF7Xh7RP
🚨 Stop feeding your world model the same boring demos. Switch to free-range play data! 🚨
Our CoRL 2026 paper finds we can optimize autonomous data collection to focus on missing action consequences, achieving up to 85x the data scaling efficiency of expert data collection!🧵
While I am very sad to miss RSS this year, my brilliant students and collaborators will be presenting five exciting papers 🧵:
1. Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement by @alexzhang_robo
https://t.co/q2RjtsA9vG
We show a simple recipe for self-improving generalist policies using a VLM verifier and an off-the-shelf generator (policy).
We started this moonshot project a year ago. Now we are excited to share our progress on robot learning from egocentric human data 🕶️🤲
Key idea: Egocentric human data is robot data in disguise. By bridging the kinematic, visual, and distributional gap, we can directly leverage human data to scale up imitation learning.
My thoughts:
1. Human data is robot data: We perceive with our eyes and act with our body and hands. If we can learn across robot embodiment, why not humans?
2. Data collection needs to be passive: Just as the Internet evolved into an unintentional data repository for AI, we envision systems that effortlessly capture embodied experiences from human activities, without humans’ conscious participation.
3. The data capturing technology is ready: Project Aria glasses by @RealityLabs@meta_aria capture all the information we need to turn human data into robot data: fisheye camera, 3D hand tracks, SLAM, … We will see more ubiquitous devices entering the consumer market.
4. The next generation foundation model will learn from embodied human data: The human sensorimotor experience is largely missing from today’s AI training data. Next-gen foundation models will understand humans from a human’s perspective, driven by large-scale embodied human data.
See @simar_kareer's thread for more details about this project.
I gave AI control of my computer and asked it to "solve homework 1 of Stanford discrete math class (Math 61DM)".
It found the problem set, downloaded Latex, solved every question, and compiled it to a PDF... in FIVE minutes.
Will any college student ever do homework again?
We are now having full conversations with Figure 01, thanks to our partnership with OpenAI.
Our robot can:
- describe its visual experience
- plan future actions
- reflect on its memory
- explain its reasoning verbally
Technical deep-dive 🧵: