I’ll be presenting our AI Coaching work at #CoRL2026—come say hi!
Poster #78 · Lone Star, 3rd floor · JW Marriott Austin
Mon, Nov 9 · 5–6:30 PM CST
Open to collaborations on:
• Large-scale human data collection
• Bringing AI Coaching to other domains and applications
• Other Research collaborations
Related ideas are welcome too!
I’m also looking for a research internship—happy to connect!
DM / [email protected]
A copilot flies the plane for you. A coach teaches you to fly on your own. What does it take to build an AI coach?
We formalize AI coaching as a non-cooperative game where the coach targets the learner's independent competence, then use model-free RL to train coaching policies.
One thing I've learned working on dexterous manipulation is how much a few millimeters can matter. A tiny change in finger position can lead to completely different contact dynamics.
Egocentric video tells us a lot about how humans interact with the world. But there's so much happening underneath that is hard to capture from pixels alone.
As the field move toward a million hours of human video, I've been thinking about what we should scale next.
I think a big part of the answer is richer, sensorized human data, with the sensing and robot hardware designed together from the start.
S1 is our first step in that direction. It's been really fun building it, and even more exciting to think about how much further we can take this.
9/9 🔗
Project page: https://t.co/FM4vOe5M65
Paper: https://t.co/2LAsmHachy
Joint work with Haimin Hu (@HaiminHu), Enlin Gu, Antonio Loquercio (@antoniloq), and Rahul Mangharam (@rahulmangharam).
A copilot flies the plane for you. A coach teaches you to fly on your own. What does it take to build an AI coach?
We formalize AI coaching as a non-cooperative game where the coach targets the learner's independent competence, then use model-free RL to train coaching policies.
8/9 🔍
We also inspect how L2C adapts assistance based on the estimated learner skill and evolving physical context.
The blending magnitude reveals two patterns: across learners, novices get more help; within a trajectory, assistance spikes near the gates, the make-or-break states for passing or colliding.
“AI agents will outperform humans at almost all jobs by 2026–2027.” - The forecast is everywhere.
So we built the exam to test that claim, on real labor-market aligned work. On the hardest tier, top agents pass 2.6%.
Meet Agents' Last Exam (ALE), a rolling benchmark measuring whether agents can actually do real jobs. 🧵👇
Agreed: Latency is now low enough to support robot inference in the cloud, and edge is where embodiment transforms and safety checks should be performed: https://t.co/SPEd2xodAR
Heard some people like wheels?😁
Humanoid robots are the ideal form of general-purpose robots (perfect for general AI and human-derived data). They can work without wheels — but they can also have wheels if they want. Whatever works.
GEN-1 plays the 🐚 shell game, trained on just 1 hr of robot data. It also generalizes to unseen objects, like @BerkayAntmen 's car keys.
Physical AI models should be capable of benchmark tasks like this one. It's interesting for the all the reasons @RhodaAI calls out -- requires visual memory, and the model must track the cups from the very start, at high frame rates.
Interestingly, GEN-1 appears to exhibit a degree of "active perception." It's subtle; the hands can sometimes appear to "follow" the cups, using its own movements to help attend to where it thinks the object should be.
Read more about GEN-1 in our blog post in the comments below ↓
For Sunday Robotics, VR teleoperation is a dead end.
The reason? Your hand is numb.
@tonyzzhao explains the physics: In VR, you miss "force closure." You accidentally apply infinite force to soft objects because you get no feedback.
Sunday's solution? Ditch the headset for a glove. 👇
@karpathy Amazing thoughts. All of human intelligence comes from a simple survival reward. Then ...
Why are fear, anger, bonding, and alliances such universally useful for surviving? Then ...
And can we simulate the emergence of fear, anger, or even language inside a simulator?