Hanjung Kim

Research Scientist Intern @nvidia GEAR | Ph.D. student @ Yonsei University | prev. @nyuuniversity

[Hanjung Kim @KimD0ing]

How can we effectively leverage human videos for robot learning by bridging the inherent embodiment gap?

We introduce UniSkill, a universal skill representation, a scalable method for learning cross-embodiment skill representations from large-scale in-the-wild video data.

1/n

[Kevin Lin @linkevin0]

Humanoids need data. Lots and lots of data.

Introducing HumanoidMimicGen: a method that automatically generates 1000s of humanoid loco-manipulation demonstrations from a single teleoperated demonstration.

[Hanjung Kim @KimD0ing]

@Zifan_Zhao_2718 great work!

[Jim Fan @DrJimFan]

I promise this will be the best 20 min you spend today! Robotics: Endgame, the sequel to my last year's Sequoia AI Ascent talk, "Physical Turing Test". I laid out the roadmap for solving Physical AGI as a simple parallel to the LLM success story. Be a good scientist, copy homework ;)

And stay till the end, more easter eggs and predictions for your polymarket!

00:30 DGX-1 origin story at OpenAI, I was there in 2016 signing with Jensen and Elon. Heading to the Computer History Museum!
01:42 The Great Parallel
03:31 Robotics, the Endgame
03:39 Why VLAs fall short
04:32 Video world models as the 2nd pretraining paradigm
06:09 World Action Models (WAM)
07:46 Strategies for robot data collection and the FSD equivalent to physical data flywheel for robot manipulation
11:06 EgoScale and the Dexterity Scaling Law we discovered recently
14:00 Physical RL: bridging the last mile
15:39 DreamDojo: an end-to-end neural physics engine for scaling RL in silico
17:00 Civilizational Technology Tree and my predictions for the near future. Spoiler: it's closer than you think.

Thanks to my friends at Sequoia for inviting me back to AI Ascent this year! I had a blast! Last year's talk is attached in the thread if you missed it.

[Max Fu @letian_fu]

Robotics: coding agents’ next frontier.

So how good are they?

We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability.

From @NVIDIA @Berkeley_AI @CMU_Robotics @StanfordAILab

https://t.co/MVcc6XWQhY

��

[Irmak Guzey @irmakkguzey]

Learning from human data requires human-like hardware. Humans use their wrists constantly, but table-top manipulators lack this flexibility.

We build upon RUKA and introduce RUKA-v2: a tendon-driven hand with a 2-DOF wrist and finger abduction/adduction 👋✌️

[Miran Heo @miran_heo]

SAM 3.1 is here 🚀
7x faster with 128 objects, without sacrificing any quality
Glad to have contributed as part of the SAM team
Special kudos to our intern @hkchengrex for the amazing contribution! 🙌

[Danfei Xu @danfei_xu]

Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.

Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling

1300+ hrs, 240 scenes, 2000+ tasks, and growing

Dataset design, findings, and ecosystem 🧵

[Hanjung Kim @KimD0ing]

@luke_ch_song @OhioState @ysu_nlp @nvidia congrats!

[Ruijie Zheng @ruijie_zheng12]

Proud to introduce EgoScale: We pretrained a GR00T VLA model on 20K+ hours of egocentric human video and discovered that robot dexterity can be scaled, not with more robots, but with more human data. A thread on 🧵what we learned. 👇

[Jim Fan @DrJimFan]

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop.

Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate.

Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution.

Our recipe is called "EgoScale":

- Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks.
- Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency.
- Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone.

The scalable path to robot dexterity was never more robots. It was always us.

Deep dives in thread:

[Jim Fan @DrJimFan]

Announcing DreamDojo: our open-source, interactive world model that takes robot motor controls and generates the future in pixels. No engine, no meshes, no hand-authored dynamics. It's Simulation 2.0. Time for robotics to take the bitter lesson pill.

Real-world robot learning is bottlenecked by time, wear, safety, and resets. If we want Physical AI to move at pretraining speed, we need a simulator that adapts to pretraining scale with as little human engineering as possible.

Our key insights: (1) human egocentric videos are a scalable source of first-person physics; (2) latent actions make them "robot-readable" across different hardware; (3) real-time inference unlocks live teleop, policy eval, and test-time planning *inside* a dream.

We pre-train on 44K hours of human videos: cheap, abundant, and collected with zero robot-in-the-loop. Humans have already explored the combinatorics: we grasp, pour, fold, assemble, fail, retry—across cluttered scenes, shifting viewpoints, changing light, and hour-long task chains—at a scale no robot fleet could match. The missing piece: these videos have no action labels. So we introduce latent actions: a unified representation inferred directly from videos that captures "what changed between world states" without knowing the underlying hardware. This lets us train on any first-person video as if it came with motor commands attached.

As a result, DreamDojo generalizes zero-shot to objects and environments never seen in any robot training set, because humans saw them first.

Next, we post-train onto each robot to fit its specific hardware. Think of it as separating "how the world looks and behaves" from "how this particular robot actuates." The base model follows the general physical rules, then "snaps onto" the robot's unique mechanics. It's kind of like loading a new character and scene assets into Unreal Engine, but done through gradient descent and generalizes far beyond the post-training dataset.

A world simulator is only useful if it runs fast enough to close the loop. We train a real-time version of DreamDojo that runs at 10 FPS, stable for over a minute of continuous rollout. This unlocks exciting possibilities:

- Live teleoperation *inside* a dream. Connect a VR controller, stream actions into DreamDojo, and teleop a virtual robot in real time. We demo this on Unitree G1 with a PICO headset and one RTX 5090.
- Policy evaluation. You can benchmark a policy checkpoint in DreamDojo instead of the real world. The simulated success rates strongly correlate with real-world results - accurate enough to rank checkpoints without burning a single motor.
- Model-based planning. Sample multiple action proposals → simulate them all in parallel → pick the best future. Gains +17% real-world success out of the box on a fruit packing task.

We open-source everything!! Weights, code, post-training dataset, eval set, and whitepaper with tons of details to reproduce. DreamDojo is based on NVIDIA Cosmos, which is open-weight too.

2026 is the year of World Models for physical AI. We want you to build with us. Happy scaling!

Links in thread:

[Zhengyi “Zen” Luo @zhengyiluo]

SONIC is now open-source!

Generalist whole-body teleoperation for EVERYONE!

Our team has long been building comprehensive pipelines for whole-body control, kinematic planner, and teleoperation, and they will all be shared.

This will be a continuous update; inference code + model already there, training code and gr00t integration coming soon!

Code: https://t.co/7u3SBxzXU9
Docs: https://t.co/HpDLkTCSMF
Site: https://t.co/D3i4KlnLLr

[Seonghyeon Ye @SeonghyeonYe]

VLAs (from VLMs) ❌ => WAMs (from Video Models) ✅

Why WAMs?
1️⃣ World Physics: VLMs know the internet, but Video Models implicitly model the physical laws essential for manipulation.
2️⃣ The "GPT Direction": VLAs are like BERT (rely heavily on task-specific post-training). WAMs are like GPT (pre-train & prompt), unlocking incredible zero-shot transfer!

What I want to see in 2026:
📈 Scaling Laws: We will see much clearer scaling laws for robotics compared to VLAs.
🤝 Human-to-Robot Transfer: Unlocking massive transfer capabilities using video as a shared representation space.
🤖 Zero-Shot Mastery: Moving from short-horizon tasks to long-horizon, dexterous manipulation without task-specific demonstrations.

We recently open-sourced the checkpoints, training and inference code.
Dive into the research! 👇
📄 Paper: https://t.co/jFEwebgyBH
💻 Code: https://t.co/4sZ5RoFmgB
🤗 HF: https://t.co/nPGoLYCPyq

[Siddhant Haldar @haldar_siddhant]

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

https://t.co/0Zi2PUPbE8

[Mahi Shafiullah 🏠🤖 @notmahi]

Why buy a robot when you can build your own?

Meet YOR, our new open-source bimanual mobile manipulator robot – built for researchers and hackers alike for only ~$10k. 🧵👇

[Jeff Cui @jeffacce]

We don't need the name of an object to pick it up; we simply need to know where it is and what it looks like.
Introducing Contact-Anchored Policies (CAPs): instead of language, we explicitly condition on contacts. Our policy learns object pickup with only 16 hours of data! 🧵

[Hanjung Kim @KimD0ing]

@redstone_hong nice work!!

[Seonghyeon Ye @SeonghyeonYe]

We just gave robots "imagination," and the results are wild. 🤯
This robot wasn't trained to untie shoes or shake hands.
It's never seen these tasks before.

It simply "dreams" the future outcome, then acts to make it real. 🧵👇

[Joel Jang @jang_yoel]

Introducing DreamZero 🤖🌎 from @nvidia

> A 14B “World Action Model” that achieves zero-shot generalization to unseen tasks & few-shot adaptation to new robots
> The key? Jointly predicting video & actions in the same diffusion forward pass

Project Page: https://t.co/qhygDzu6NY
🧵 (1/10)

[Jim Fan @DrJimFan]

https://t.co/Npar79SvUh