Sirui Chen

PhD in Stanford CS, Prev Undergrad at HKU. Interested in robotics Currently interning at Amazon FAR

[Sirui Chen @eric_srchen]

Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction!
By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like carrying a box upstairs. Code & data open-sourcing soon! 📦 🪜
Paper: https://t.co/aHLo5pPZQr
Website: https://t.co/SRUsbC6Y9b

[Austin Patel @austinapatel]

LLMs learn new tasks in-context. It’s time robots do the same 🤖

Introducing Behavior Prompting: human shows one demo, and the robot adapts immediately. Turns out robot demos are great in-context prompts!

And the magic: no paired human2robot data needed

https://t.co/TNikpJT61S (1/8)

[Hojune Kim @HojuneKim]

What if a phone scan is all you need to teach a humanoid a new skill, one that generalizes to scenes it's never seen?

Introducing 🦿LEGS🦿: a photorealistic loco-manipulation simulator. No teleop; policies deploy zero-shot on a Unitree G1 🤖

https://t.co/Ubfep7hP7l 👇

[Sirui Chen @eric_srchen]

When humanoids have all features that other platform have but have advantage that other platform don't, the benefit of humanoids shall stand out without doubt

[Qingzhou Lu @Axell_wppr]

Humanoids should take on the heavy lifting jobs for humans. But can full-size humanoids handle heavy-payload teleoperation from noisy VR inputs?

Excited to introduce our work, HEFT: Heavy-Payload Full-size Humanoid Teleoperation.

HEFT tracks human intent from raw, noisy VR signals and enables real-world teleoperation with payloads up to 24 kg on L7, a 175 cm, 65 kg full-size humanoid.

Website & more demos: L7 heavy-payload teleop + G1/L7 high-dynamic tracking
https://t.co/fFgSWgpA7V

G1 & L7 training code/checkpoints:
https://t.co/uGimX29xyU

[Yen-Jen Wang @wangyenjen]

Our whole-body tracker in VLK (https://t.co/PnvpCDW4fi) is powered entirely by SceneBot. Having been one of the first users of SceneBot, I can confidently say its capabilities are truly impressive. VLK predicts both whole-body kinematics and contact states, while SceneBot faithfully tracks and executes them through its global whole-body motion tracking with contact-aware control. It has been an incredibly strong foundation for bringing VLK to the real world. Huge shoutout to the SceneBot team! 🚀

[Dantong Niu @Dantong_Niu]

Excited to share T-Rex: Tactile-Reactive Dexterous Manipulation 🦖🤖

Touch is fundamental to human dexterity, yet most Vision-Language-Action (VLA) models either ignore tactile feedback or lack the ability to react to high-frequency contact signals.

In this work, we tackle both the data and architectural challenges of tactile-reactive dexterous manipulation.

🦖 A 100-hour tactile-synchronized dexterous manipulation dataset with 7,700+ trajectories, 22 motor primitives, and 200+ everyday objects.

🦖 A tactile-reactive MoT architecture with spatial-temporal tactile encoding and asynchronous high-frequency tactile refinement.

🦖 A scalable training recipe combining 22,889 hours of human egocentric pretraining with tactile-grounded robot mid-training.

Across 12 real-world contact-rich manipulation tasks, T-Rex achieves over 30% higher average success rate than the strongest baseline.

We are fully open-sourcing the dataset, models, teleoperation stack, training code, and inference pipeline.

🌐 Project: https://t.co/AiHKRR8YXU
📄 Paper: https://t.co/mXY2UNLlqc
💻 Code: https://t.co/7skCxUtwKC
🤗 Dataset: https://t.co/uNwW8dcRZL
🧵 Thread ↓

[Wenli Xiao @_wenlixiao]

Autoresearch just left the sandbox and entered the embodied world.

We are excited to introduce 𝐄𝐍𝐏𝐈𝐑𝐄: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop:

reset the environment → search the literature → implement ideas and build the infra → train and deploy → self-verify → analyze the logs and rewrite the code → repeat, until the policy is reliable in the real world. No human in the loop.

Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties.

We envision the bottleneck in robotics shifting — from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own.

🔗 https://t.co/3tL2ArGo3v
From @NVIDIA @CMU_Robotics @Berkeley_AI
🧵

[Ruiqian Nai @RuiqianNai]

Humanoid loco-manipulation from synthetic data!

[Yen-Jen Wang @wangyenjen]

How can we scale perception-based humanoid learning without collecting massive humanoid teleoperation data?

🚀 Excited to finally share VLK!

What excites me most about VLK is that it reframes data collection as a data generation problem. Instead of relying on expensive humanoid teleoperation, we automatically generate synchronized vision, language, and whole-body kinematics from reconstructed real-world scenes.

Making this vision a reality required bridging three fundamental challenges:

👀 Perception: Bridging the RGB sim → real gap through visual domain randomization and motion blur mitigation during both training and deployment.

🤖 Embodiment: Bridging the kinematics → dynamics gap with real-time VLA deployment, test-time RTC, and SceneBot, enabling seamless deployment on a real humanoid.

🌍 Environment: Bridging the real-world → synthetic gap to enable scalable Vision-Language-Kinematics data generation through scene reconstruction and interaction synthesis.

It has been an amazing journey working with such an incredible team. For a complete walkthrough of the project, check out @jiaman01's thread below 👇

🌐 Project: https://t.co/PnvpCDW4fi
📄 Paper: https://t.co/DPe20ilXm7
🎦 Video: https://t.co/BivXCxkzcq

Huge thanks to my amazing collaborators @jiaman01 @eric_srchen @TakaraTruong @ Pei Xu, and to our advisors @pabbeel @rocky_duan @KoushilSreenath @akanazawa @carlo_sferrazza @GuanyaShi @ckarenliu.

[Sirui Chen @eric_srchen]

@alfie14x For terrain it should be pretty robust, as long as terrain can be described by a 2.5D height map the algorithm should be able to reconstruct it, some limitations may be ladders where stair above may cover stair below

[Sirui Chen @eric_srchen]

For many tasks, data demand is high, but teleop is hard. We propose VLK, the first whole-body humanoid VLA policy trained solely on synthetic data. We show that it can autonomously navigate and perform object pick-and-place using language commands.

[Jiaman Li @jiaman01]

🤖 How can we scale up humanoid robot learning?

Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed!

Website: https://t.co/Er9BfeYfeM

[Robots Digest 🤖 @robotsdigest]

VLK asks a simple question:

Can we train humanoid VLA policies without collecting expensive teleoperation data?

Instead of relying on human demonstrations, the paper reconstructs real indoor environments with 3D Gaussian Splatting, synthesizes humanoid interactions inside them, and automatically generates paired vision, language, and whole-body trajectories.

[Tyler Lum @tylerlum23]

🤖 How can we teach dexterous robots to perform precise, contact-rich assembly?

Introducing Play2Perfect: first learn to play with objects, then perfect the policy for tight insertion, multi-part assembly, and screwing.

Sound on! 🔊

🧵👇

[Sarvesh Patil @servo97]

Interaction with the real world is the major bottleneck in robot learning. So what would robot RL look like if we didn’t need to limit compute per interaction? Our latest work, Off-Policy Generative Policy Optimization (OGPO, accepted to ICML26) embarks on answering this question (spoiler alert: when done correctly, it helps massively!).
🧵(1/N)

[Jiashun Wang @JiashunWang]

Reference motions are often used as trajectories to track or teachers to distill. We explore a different way of learning from them.

I am excited to share our work, Generalizing from References (GfR), to appear at RSS 2026, as a follow-up to our previous HIL work.

Using a unified multi-task RL framework, we jointly train reference-guided imitation and goal-driven RL within a single end-to-end policy.

No distillation.

No RL fine-tuning.

Just one policy, trained end-to-end, that learns from references and generalizes beyond them.

Rather than treating reference motions as trajectories to track, distill, or follow, we use them to shape behavior while allowing RL to explore and adapt beyond the references.

In the following example, without human joystick control, the robot can autonomously compose learned skills using only task goals.

🌐 https://t.co/pdMWBWgtCY

🤖 Things beyond locomotion coming soon.

[Sirui Chen @eric_srchen]

Contact aware whole body control has always been a missing piece!

[Runyi (Ingrid) Yu @Runyi_Ingrid_Yu]

🎉 Excited to share our new work: OmniContact🎉

We introduce a framework built on "Contact Flow" to tackle the challenges of generalizable loco-manipulation and long-horizon task planning for 🤖.

✅ Code, Models, and dataset are all avaliable!

🌐: https://t.co/jovO19xQv2

[Shibo @ShiboZhaoSLAM]

Excited to share what we've been working on! Huge kudos to Sirui for leading this! The single-policy approach for diverse scene interaction is just elegant. It's so rewarding to see it handle all those different motions with one unified solution. Honored to be a co-author on this journey. ❤️

[Shibo @ShiboZhaoSLAM]

Huge Congrats, Sirui! It’s been a pleasure collaborating with you. I really appreciate all the hard work you’ve put into making this project a reality. The single-policy paradigm for diverse scene interactions is incredibly exciting！I’m glad SuperOdometry could contribute to the system. Looking forward to seeing what the community builds on top of SceneBot!

[Yan Chang @Dr_YanChang]

How can robots learn dexterous manipulation from human demonstrations at scale?
Excited to share CHORD: Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration.
CHORD learns from human demos by focusing not only on where contact happens, but how that contact moves the object through force and torque guidance.
This unified contact-wrench representation carries human manipulation skills across diverse behaviors, long-horizon tasks, whole-body embodiments, and real-world hardware.
We evaluated CHORD on large-scale, long-horizon, contact-rich tasks paired with human demonstrations, spanning rigid, articulated, and multi-object manipulation.
At scale:
* 82.12% average success across 1,831 tasks
* 90.77% whole-body manipulation success
* 4,739 sim-ready dexterous manipulation benchmark
* Transfer to real dexterous hands

Project page: https://t.co/cAnHaJUy6P
Tech report: https://t.co/oWMzszwrgw
Code will be released soon as part of Video to Data repo https://t.co/spdPL4Gxt6, our end-to-end pipeline for converting human demonstration videos into simulation-ready assets and physics-grounded robot training data.
Huge thanks to amazing contributors: @zhu_xinghao , Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, @michaelv03, Wei Liu, @TingwuWang , Xingye (Dennis) Da, @zhengyiluo, Vishal Kulkarni, @sNaema, @yukez, @DrJimFan, @bowenwen_me, @danfei_xu, @SohaPouya, @Dr_YanChang.
#Robotics #PhysicalAI #DexterousManipulation #RobotLearning #NVIDIA

[clankr @clankrmedia]

WBC is cool but it doesn't have real world functionality.

So researchers thought, what if we combine both of them?

This is how SceneBot was born.
A whole-body controller that can actually make contact and pull off real-world tasks.

The long-horizon clip says it best, carrying a box WHILE climbing stairs in one shot.

Here is the problem :
General motion trackers walk, run, dance, kick beautifully with a single policy.

But the moment a robot touches a box or a stair, pure kinematics breaks down.

So what changes here:
🔵 They add per-link contact labels on top of the reference motion.
🔵 The policy gets told which hand grabs the box and which foot lands on the stair.
🔵 ONE single policy now covers both free-space motion and heavy contact work.

A dataset with motion plus full scene interaction barely exists.

So they reconstruct the scene backwards from plain human motion.

Given a retargeted motion, they infer where contacts happened and rebuild a plausible box or stair around it.

The numbers back it up hard.
On object tasks SONIC sits near 5% success while SceneBot hits 95%.

We highly recommend trying the simulation demo on their project page.
They basically packed almost everything they built into a SINGLE demo.

[Sirui Chen @eric_srchen]

This work won't be possible without amazing collaborators: @ShiboZhaoSLAM, @zhenkirito123, and @jiaman01. Advisors: @GuanyaShi and @ckarenliu, as well as support from the Amazon FAR team( @wangyenjen, @xupei0610)!

[Sirui Chen @eric_srchen]

Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction!
By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like carrying a box upstairs. Code & data open-sourcing soon! 📦 🪜
Paper: https://t.co/aHLo5pPZQr
Website: https://t.co/SRUsbC6Y9b

[Sirui Chen @eric_srchen]

We build an interactive demo with SceneBot policy tracking motion from a motion matching engine. Now playable in desktop website: https://t.co/qkszMNwt44