Introducing TAX3D, in which we extend relative-placement methods to generalizable deformable manipulation! #CoRL2024 (1/🧵)
Our approach generalizes to:
- Diverse unseen objects
- Diverse unseen configurations
- Multimodal placements
Finally back in Singapore 🇸🇬!
Excited to start my first day at @NUSComputing and joint appointment with @ASTARsg !
I’m excited to contribute to Singapore’s Embodied AI landscape, which has grown tremendously since I left for my PhD four years ago. If you’re working on Physical AI in Singapore, feel free to reach out, I’d love to chat about potential collaborations and opportunities!
I'm attending #RSS2026 for a couple papers/talks and as an RSS Pioneer! Please reach out to chat and/or come to the following:
- Monday 9:40 AM: Spotlight pres on detecting side effects at the Rethinking Safety Workshop by @ryanlindeborg (https://t.co/RNZR3P7owE)
- Tues 4-5PM: Come visit my RSS Pioneers Poster! @RSSPioneers
- Wed 3:15PM: Robometer Reward Fxn talk + poster after by @yigitkkorkmaz and @aliangdw (https://t.co/nd1jpgRM0C)
- Thurs 3:15PM: TMRL pre-training for post-training talk + poster after by @matthewh6_ (https://t.co/goYhXBw7TZ)
- Friday 9:30AM: Giving an invited talk at SemRob Workshop (https://t.co/aseCY9NRCj)
- Friday 5:30PM: Invited talk at the Diffusion Workshop (https://t.co/jpKl1zmTWz) on TMRL
RL can't teach an LLM to solve problems it never solves. Robot policies suffer from exactly the same limitation.
The fix turned out to be almost embarrassingly simple: train the policy with diffusion noise during pre-training.
That's it.
The policy covers a much wider action distribution, RL finally has somewhere to search, and we fine-tune VLAs on real robots in under an hour.
Introducing TMRL.
🧵(1/9)
I'll be presenting this work at #RSS2026 next week! For more details, check out our paper and project website:
📄 https://t.co/SGusFjrR2r
🌐 https://t.co/UBNvaA4pAm
Huge shoutout to my amazing collaborators @Jesse_Y_Zhang and Anusha Nagabandi, and to my advisor @abhishekunique7 for the guidance and support throughout every stage of this project! None of this would have been possible without them.
(9/9)
Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations.
Can we improve them without any additional data collection on the real robot?
In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience:
https://t.co/hbhId2qbB1
🧵 (1/10)
Boo! 👻 GHOST has been accepted to #RSS2026!
Can we learn manipulation skills from human video (without action retargeting)?
Yes! GHOST learns generalizable manipulation skills by training a hierarchical policy with an embodiment-agnostic sub-goal predictor + embodiment-specific controller.
🧵(1/7)
Want to finetune generative control policies (flow/diffusion) super efficiently?
Check out our new work OGPO - which uses off-policy TD-learning with PPO-based policy extraction to get stable, efficient updates for policy finetuning.
@servo97 and @max_simchowitz have done a really fantastic job with a deep scientific study on the behavior of this algorithm and what it takes to get it to work in practice!
Paper: https://t.co/NmjY1fPmEA
Website: https://t.co/oz97Qkqzf3
Code: https://t.co/6PkHUH8uam
Interaction with the real world is the major bottleneck in robot learning. So what would robot RL look like if we didn’t need to limit compute per interaction? Our latest work, Off-Policy Generative Policy Optimization (OGPO, accepted to ICML26) embarks on answering this question (spoiler alert: when done correctly, it helps massively!).
🧵(1/N)
How can robots place objects precisely when there are tight tolerances and changing object geometry, for example, reliably inserting a connector into a socket?
Introducing TAX-DPD, a hierarchical point-diffusion method for object-centric goal prediction. TAX-DPD achieves 80-100% accuracy on high precision insertion tasks on the NIST task board, and also only a 3% error rate on the RPDiff object placement benchmark that requires generalizing over object geometry. #ICRA2026
Project: https://t.co/kBpLSBsJbn
Paper: https://t.co/LprP4lF9s4
Code: https://t.co/bPsCnoOzWN
Real-world RL is still too brittle and data-hungry for long-horizon, contact-rich tasks.
We introduce Simulation Distillation (SimDist), which turns large-scale simulated experience into reusable world-model priors for rapid real-world adaptation.
By combining online planning with dynamics adaptation, SimDist achieves high success rates on tasks requiring precision, force, and reactivity.
Play with our interactive visualization to see for yourself: https://t.co/qFGNySxdAl
(1/n)
Launching my research group, MAGIC (Manipulation and General Intelligence Control) Lab @NUSComputing, Singapore!
We focus on building the next generation of human-centric models for robotic manipulation — deployable safely, reliably, and easily in the real world. Our research spans MLLM reasoning, 3D vision, robot learning, simulation, dexterous manipulation, and cross-embodiment learning.
Interested in joining? Sign up here and I'll send a reminder email: https://t.co/9lEQnFERuh
Robotics models often struggle outside controlled environments. Ours is built to work in real ones.
Today we're launching MolmoAct 2, which can assist with a host of chores & lab tasks, plus the MolmoAct 2-Bimanual YAM dataset—the largest open robotics dataset of its kind. 🧵
[Major life updates] 🎉
After 4 incredible years of my PhD at @UW@uwcse with @fox_dieter17849 and @RanjayKrishna, I'm joining @NUSComputing as an Assistant Professor this August, under the Presidential Young Professorship scheme!
More details 🧵👇
4 years have been simply amazing! I’m happy to share that I have successfully defended my PhD!
Thank you to everyone who came to support me, and most importantly, to my thesis committee, advisors, collaborators, friends, and family for being part of this journey.
Great to have @Jesse_Y_Zhang visiting us @IRVLUTD today!
He shared his journey toward generalist robotics reward models (RoboCLIP, ReWiND, Robometer), followed by a great buffet with the lab.
🤖How do we evaluate robots?
Fixed benchmarks.
Same tasks.
Predefined success.
‼️But this doesn’t tell us what robots actually understand.
➡️We built **RoboPlayground**.
✨TLDR: Evaluation should be a space anyone can define, not a fixed benchmark.✨
🧵1/n
#robotics#ai#robot
We’re releasing OmniReset, a framework for training robot policies using large-scale RL and diverse resets for contact-rich, dexterous manipulation.
OmniReset pushes the frontier of robustness and dexterity, without any reward engineering or demonstrations.
Try the policies yourself in our interactive simulator! https://t.co/3hW3nYx2vD
(1/N 🧵)