Excited to share Heave-Ho, my co-first-author paper with @SihengZhao and @hanjiang00!
๐ค๐ค๐ค Just like our humanoids, we got more done together. Huge thanks to my advisor @yuewang314 and everyone who helped and collaborated with us along the way!
https://t.co/pnyvl836v0
Excited to share Heave-Ho, my co-first-author paper with @SihengZhao and @hanjiang00!
๐ค๐ค๐ค Just like our humanoids, we got more done together. Huge thanks to my advisor @yuewang314 and everyone who helped and collaborated with us along the way!
https://t.co/pnyvl836v0
๐ Introducing ๐๐น๐ฒ๐ -๐ฅ๐ฆ๐: ๐๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ๐ถ๐ป๐ด ๐ฅ๐ฒ๐ฐ๐๐ฟ๐๐ถ๐๐ฒ ๐ฆ๐ฒ๐น๐ณ-๐๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐บ๐ฒ๐ป๐ ๐ผ๐ณ ๐๐ฟ๐ผ๐ป๐๐ถ๐ฒ๐ฟ ๐๐ด๐ฒ๐ป๐๐
We're curious: ๐๐ข๐ฏ ๐ข๐ฏ ๐๐ ๐ข๐จ๐ฆ๐ฏ๐ต ๐จ๐ฆ๐ต ๐ฃ๐ฆ๐ต๐ต๐ฆ๐ณ ๐ข๐ต ๐ช๐ต๐ด ๐ซ๐ฐ๐ฃ ๐ฃ๐บ ๐ณ๐ฆ๐ธ๐ณ๐ช๐ต๐ช๐ฏ๐จ ๐ฉ๐ฐ๐ธ ๐ช๐ต ๐ธ๐ฐ๐ณ๐ฌ๐ด? ๐
We tested ๐ญ๐ด frontier models on ๐ฎ๐ฐ tasks, from ๐ฟ๐ผ๐ฏ๐ผ๐ ๐ฐ๐ผ๐ป๐๐ฟ๐ผ๐น and ๐ฏ๐ ๐ฟ๐ฒ๐ฐ๐ผ๐ป๐๐๐ฟ๐๐ฐ๐๐ถ๐ผ๐ป to ๐ฟ๐ฒ๐ฎ๐๐ผ๐ป๐ถ๐ป๐ด and ๐๐ถ๐๐๐ฎ๐น ๐ฐ๐ฟ๐ฒ๐ฎ๐๐ถ๐ผ๐ปโ๐ญ๐ด๐ด runs and nearly ๐ฏ,๐ด๐ฌ๐ฌ rounds of self-improvement. Agents could revise their code, tools, prompts, and workflows while their model weights stayed fixed.
A few observations I found particularly interesting:
๐ ๏ธ ๐๐๐ถ๐น๐ฑ๐ถ๐ป๐ด ๐ฟ๐ฒ๐๐ฒ๐ฎ๐น๐ ๐บ๐ผ๐ฟ๐ฒ ๐๐ต๐ฎ๐ป ๐ฎ๐ป๐๐๐ฒ๐ฟ๐ถ๐ป๐ด.
The largest gains came when agents built programs, robot policies, or pipelines. Models starting from the same solution ended up far apart.
๐ ๐๐ฒ๐ฒ๐ฑ๐ฏ๐ฎ๐ฐ๐ธ ๐ฐ๐ฎ๐ป ๐ฏ๐ฒ ๐๐ต๐ฒ ๐ฏ๐ผ๐๐๐น๐ฒ๐ป๐ฒ๐ฐ๐ธ.
Open-weight models made progress with graded feedback but often stalled when they received only success or failure. On RoboCasa, median validation gains were 51.2 points for closed models versus 5.6 for open-weight models.
๐งฑ ๐๐ผ๐ผ๐ธ๐ถ๐ป๐ด ๐ฟ๐ถ๐ด๐ต๐ ๐ถ๐ ๐ฎ ๐น๐ผ๐ ๐ฏ๐ฎ๐ฟ.
Some reconstructed scenes looked convincing but failed physical stability or object-level checks. The gap between closed and open-weight models was often largest where correctness could be tested strictly.
โฑ๏ธ ๐ ๐ผ๐ฟ๐ฒ ๐๐ผ๐ธ๐ฒ๐ป๐ ๐ฑ๐ถ๐ฑ๐ปโ๐ ๐บ๐ฒ๐ฎ๐ป ๐ฏ๐ฒ๐๐๐ฒ๐ฟ ๐ฟ๐ฒ๐๐๐น๐๐.
Token use and performance were essentially uncorrelated across the evaluated runs. Several of the strongest models also used fewer tokens.
And the workflow edits themselves helped: in 6 of 7 paired experiments, agents allowed to rewrite their own harness achieved higher test scores than those with a fixed harness.
Glad to share this work with Jingdong Zhang, Jionghao Wang, Zhengming Yu, Yancheng Cai, Wei Chow, Chris Choy, Zhiyang (Frank) Dou, Hang Hua, Jiachen li, Zhixuan Liu, Chen Wang, Songlin Wei, Siyuan Yang, Yu Yuan, Xiaohang Zhan, Yucheng Zhang, Shuhong Zheng, Xin (Shane) Li, Lingjie Liu, Wojciech Matusik, Wenping Wang.
๐ฅ Work done by
@TAMU, Adobe, @CarnegieMellon, @MIT, MIT-IBM Watson AI Lab โ IBM Research, @NUSingapore, NVIDIA, @LifeAtPurdue, @Cambridge_Uni, @Penn, @USC, @UTAustin, @UofT.
๐ค Flex-RSI welcomes community contributionsโnew tasks, models, and evaluators are all welcome.
๐ ๐ฃ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐: https://t.co/QmAbKfo4wy
๐ Report: https://t.co/6cWMiQraqU
#AIAgents #SelfImprovement #Robotics #RSI #AGI #GenAI #LLM #Agents #TAMU #EmbodiedAI #Vision
Meet GR00T 2, our new generation of world-action models (WAMs): it imagines how the physical world will unfold and generates the actions that get it there. One model runs on parallel grippers, dexterous hands, and humanoids.
https://t.co/32Z1plSGQg
Honored to see this.
Grateful to the GR00T / GEAR team for the chance to contribute to a release at this scale. The model, the data, and the evals are a much bigger effort than any one piece.
Second time being part of something this large, after Nemotron 3.5 Super VL. Still a bit unreal.
Meet GR00T 2, our new generation of world-action models (WAMs): it imagines how the physical world will unfold and generates the actions that get it there. One model runs on parallel grippers, dexterous hands, and humanoids.
https://t.co/32Z1plSGQg
@Michael_J_Black Thanks. I totally agree. Thatโs the real question. A result that beats a large model is out of date as soon as a new model comes out. Understanding why something works stays useful. Maybe reviewers should ask โwhat did we learn?โ more than โis it the best or SOTA?โ
I'm looking for one emergency reviewer for my @iclr_conf 2027 batch! If you have experience in VLA / latent reasoning, please DM me. Thank you!
#ICLR2027
๐ค What if robots could reason not just in words, but through the dynamics of the physical world?
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence across embodiments.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
Honored to be part of this.
First time contributing to a model release at this scale. Grateful to the Nemotron team โ the model, the data, and the evals are a much bigger effort than any one piece.
Our piece was SpatialClaw: used to build the agentic vision training data for video understanding.
๐ Paper: https://t.co/NuIZ095EYR
๐ป Code: https://t.co/H6Z2AFOXij
๐ Report: https://t.co/p2mYzgpzJa
Honored to be part of this.
First time contributing to a model release at this scale. Grateful to the Nemotron team โ the model, the data, and the evals are a much bigger effort than any one piece.
Our piece was SpatialClaw: used to build the agentic vision training data for video understanding.
๐ Paper: https://t.co/NuIZ095EYR
๐ป Code: https://t.co/H6Z2AFOXij
๐ Report: https://t.co/p2mYzgpzJa
Meet NVIDIA Nemotron 3.5 Super VL
Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents.
120B total parameters with 12B active ๐งต
Honored to be part of this.
First time contributing to a model release at this scale. Grateful to the Nemotron team โ the model, the data, and the evals are a much bigger effort than any one piece.
Our piece was SpatialClaw: used to build the agentic vision training data for video understanding.
๐ Paper: https://t.co/NuIZ095EYR
๐ป Code: https://t.co/H6Z2AFOXij
๐ Report: https://t.co/p2mYzgpzJa
Update: SpatialClaw was used in the agentic vision training data for NVIDIA Nemotron 3.5 Super VL. Grateful to be part of that effort.
14,169 trajectories, 81,366 actions. Details: https://t.co/zt2Hl2ZrMZ
Honored to be part of this.
First time contributing to a model release at this scale. Grateful to the Nemotron team โ the model, the data, and the evals are a much bigger effort than any one piece.
Our piece was SpatialClaw: used to build the agentic vision training data for video understanding.
๐ Paper: https://t.co/NuIZ095EYR
๐ป Code: https://t.co/H6Z2AFOXij
๐ Report: https://t.co/p2mYzgpzJa
SpatialClaw accepted to #NeurIPS2026 ๐
Even GPT-6 Astra gets better with the right action interface.
Same training-free harness, no model-specific tuning, 20 spatial benchmarks:
GPT-6 Astra: 71.3 โ 77.4 (+6.1)
Opus 5: 58.5 โ 73.5 (+15.0)
GPT-6 Sol: 61.2 โ 72.9 (+11.7)
Qwen3.8-27B: 59.8 โ 67.3 (+7.5)
Astra already looks โspatial.โ A persistent Python kernel on top of it still wins.
Code is the action interface: the VLM writes one cell per step, inspects SAM / depth / geometry outputs, revises, then answers.
๐ Paper: https://t.co/NuIZ095EYR
๐ป Code: https://t.co/H6Z2AFOXij
๐ Website: https://t.co/HbdASfTosl
@NVIDIAAI