Haoran Geng

CS PhD at @Berkeley_AI. Prev: @Stanford, @PKU1898. Robotics, RL, 3D Vision

[Haoran Geng @HaoranGeng2]

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate?

🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch.

🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks.

Code is fully released; check out our:
Homepage: https://t.co/I8kVJO3XPw
Paper link: https://t.co/BFqpOwXYbT
Github: https://t.co/345eTpTx0y

[Siyuan Huang @siyuanhuang95]

Humanoid robots shouldn't just follow pre-defined movements—they should perform! 🤖✨

Introducing UniAct: A unified model for multimodal motion generation and action streaming.

Most humanoids are limited to pre-designed moves. UniAct changes the game by allowing robots to generate live action sequences from:

📝 Text instructions 🎶 Music rhythms 📉 Spatial trajectories 🔄 Cross-modal signals

Whether it’s dancing to a beat or following a complex path, UniAct brings humanoid robots to life in real-time. 🚀

🔗 Project: https://t.co/ecAZk8ta8i
📄 Paper: https://t.co/ZfriCthUtz

[Haoran Geng @HaoranGeng2]

Excited to see what @SharpaRobotics has achieved recently. Proud that we shaped the ViTacFormer prototype—even before the hand was fully product-ready, we were already making a hamburger (and doing many other tasks).

My takeaway: how you fuse vision + touch matters a lot. With clean, high-quality teleop data, you can get impressive generalization with even ~50 demos.

[Simon Kalouche @simonkalouche]

This is the most dexterous task I’ve seen a humanoid do so far.

Fully autonomous powered by Sharpa’s CraftNet (VTLA) — using tactile feedback to continuously fine-tune the last-millimeter interaction.

[AK @_akhaliq]

Large Video Planner Enables Generalizable Robot Control

[Haoran Geng @HaoranGeng2]

@karanjagtiani04 Human data is actually much more scalable than robot data. Large part of our training data are in the wild human data, which i believe is quite scalable.

[Haoran Geng @HaoranGeng2]

This might be my "aha moment" of 2025:

With our new robotics foundation model, Large Video Planner, we train a robot planner from large-scale video data. It works so well that we can use it directly for robot planning.

Two moments really blew my mind:
First: right after our model training, I fed in an image of my hand and my MacBook and asked it to close the laptop—when the Apple logo appeared exactly as the lid came down, I couldn’t help but feel impressed (and excited).
Second demo: picking up the brush — check the 3D consistency. Even the brush shadow is remarkably accurate, and it can even infer what the Franka arm (at the corner) should look like.

[Haoran Geng @HaoranGeng2]

@bruk_phi We actually use a lot of human dataset, large part of the training dataset are human data, and we also use human hand as an intermediate representation for robot planning.

[Haoran Geng @HaoranGeng2]

Also check more intro here: https://t.co/MZdnUlvN3K
Do try our model and check more information here:
https://t.co/XDwZug3km2

[Boyuan Chen @BoyuanChen0]

Introducing Large Video Planner (LVP-14B) — a robot foundation model that actually generalizes. LVP is built on video gen, not VLA. As my final work at @MIT, LVP has all its eval tasks proposed by third parties as a maximum stress test, but it excels!🤗
https://t.co/wjD54YFK3k

[Haoran Geng @HaoranGeng2]

Because of the model’s remarkably accurate 3D consistency and human-hand prediction, we can use it directly for robotics planning—and it already pulls off a bunch of amazing dexterous tasks: zero-shot and, in some cases, first-ever.

It also makes me believe even more that human-centric, in-the-wild data has huge potential to further boost robotics pretraining.

[Haoran Geng @HaoranGeng2]

Been working on this for a long time—Large Video Planner is finally out. We found that large-scale video pretraining can actually teach models surprisingly accurate physics.

With strong camera consistency and realistic hand shaping, we can use the model directly for robotics planning. When I tried using our dexterous hand to pull off extremely challenging zero-shot tasks—often for the first time—made it clear just how capable this model is.

[Boyuan Chen @BoyuanChen0]

Introducing Large Video Planner (LVP-14B) — a robot foundation model that actually generalizes. LVP is built on video gen, not VLA. As my final work at @MIT, LVP has all its eval tasks proposed by third parties as a maximum stress test, but it excels!🤗
https://t.co/wjD54YFK3k

[lily zhang @lily_gpupoor]

🏆 Announcing #IROS RoboGen Best Open Source Award: RoboVerse @FeishiWang @siyuanhuang95 @yuewang314

[Haoran Geng @HaoranGeng2]

@carlo_sferrazza @UTAustin @utmechengr @texas_robotics Congrats Carlo!!!🔥

[Haoran Geng @HaoranGeng2]

Huge thanks to all the contributors and the great RoboVerse team: https://t.co/qs52J5zmCo
All our code is open-source at https://t.co/tfWU6heOfF

[Siyuan Huang @siyuanhuang95]

🎉🎉🎉 We won the champion in the solo dance contest at the first World Humanoid Robot Games, partner with @UnitreeRobotics ! Here is the full video!

Training the robot to perform a long-term dancing (2:30 mins) with stability, smoothness, and agility is much more challenging than we expected. The robot needs to dance with the rhythm, keep global position, move dynamically and cannot fall.

You cannot cherry pick on the playing field. More technical details will be released in the future.

[Haoran Geng @HaoranGeng2]

@QianqianWang5 Congrats!!!

[Ronald van Loon @Ronald_vanLoon]

Introducing ViTacFormer: Next-Level Dexterous Manipulation with Active Vision and High-Resolution Touch
by @HaoranGeng2

#AI #Robotics #MachineLearning #ArtificialIntelligence #ML #Innovation

cc: @sallyeaves @amuellerml @marcusborba

[Haoran Geng @HaoranGeng2]

@adcock_brett FYI, check out our original post here:
https://t.co/elCtqMlUSf

[Haoran Geng @HaoranGeng2]

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate?

🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch.

🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks.

Code is fully released; check out our:
Homepage: https://t.co/I8kVJO3XPw
Paper link: https://t.co/BFqpOwXYbT
Github: https://t.co/345eTpTx0y

[Haoran Geng @HaoranGeng2]

Thank you so much, @adcock_brett, for featuring our new work, ViTacFormer!

Generalizable and robust manipulation remains a long-standing and challenging goal in robot learning—we’re excited to keep pushing the boundaries.

More exciting things are on the way—stay tuned! 🚀

[Brett Adcock @adcock_brett]

@OfficialLoganK UC Berkeley researchers introduced ViTacFormer, a unified visuo-tactile pipeline for robot manipulation

It fuses high-resolution visual and tactile data using cross-attention and enables multi-fingered hands to perform precise, long-horizon tasks

[Jitendra MALIK @JitendraMalikCV]

Again the power of tactile sensing and multi-finger hands comes through. This is the future of dexterous manipulation!

[Humanoid Scott @GoingBallistic5]

There is a raging debate over sensory modes and redundancy

How much is enough and is sensory overload an issue

Perhaps the key is redundant modes are the fabric that hold actions together to solve long term planning

Think fascia

[Haoran Geng @HaoranGeng2]

This work wouldn’t have been possible without the incredible support from great collaborators @cosm_para13983 and @KaifengZhang4, and my amazing advisors @JitendraMalikCV and @pabbeel. Thank you all! 🙏

Code is fully released; check out our:
Homepage: https://t.co/I8kVJO3XPw
Paper link: https://t.co/BFqpOwXYbT
Github: https://t.co/345eTpTx0y

[Haoran Geng @HaoranGeng2]

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate?

🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch.

🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks.

Code is fully released; check out our:
Homepage: https://t.co/I8kVJO3XPw
Paper link: https://t.co/BFqpOwXYbT
Github: https://t.co/345eTpTx0y

[Haoran Geng @HaoranGeng2]

We then explored the full capabilities of our system—and found it can handle super long-horizon tasks end-to-end.

🍔 Sit back and enjoy the hamburger-making policy in action!