Amish Sethi✈️CoRL

CS PhD @Harvard @KempnerInst | Prev @Penn @GRASPlab @AfterQuery | robot learning, world models | @NSF GRFP 2026

[Heng Yang @hankyang94]

Happy Halloween from the Harvard Computational Robotics Group! ����🤖

Introducing GRASP: gradient-guided sampling for physics-based motion retargeting to dexterous hands and humanoids.

Human demonstrations are a powerful source of robot skills. But kinematically retargeted motions can look right while requiring contacts and dynamics the robot cannot realize. Downstream RL then has to learn the skill while tracking a physically inconsistent reference.

GRASP refines these references in simulation before policy learning. Better references help RL learn contact-rich motion tracking more effectively, as we demonstrate in pen spinning and humanoid–object interaction.

On real hardware, we demonstrate open-loop trajectory replay on Sharpa hands and learned tracking policies on a Unitree G1. We hope this is a step toward scaling motion tracking to more diverse contact-rich tasks.

1/3

[Amish Sethi✈️CoRL @amish_sethi04]

Agree that cost and tracking is an issue in itself which still isn't perfect. But another big reason is transfer! Getting a robot hand to reproduce the same contacts and forces as the glove is nontrivial. Some wearable hand-tracking rigs add bulk to address this (e.g., rigid finger linkages that constrain your movements to match the robot hand’s) which make them harder to use for natural manipulation. UMI keeps it simple and closer to what a robot can execute (and parallel jaw grippers seem to be able to conduct the vast majority of tasks). But i totally agree that we need to explore both! And work like DexUMI is already extending this approach to robot hands: https://t.co/QHcWf94b28

[JoHn @JOhN07x2]

Robot data collection that's one button, no base stations, no scene calibration, and episodes land in LeRobot format while you keep recording. 10/10 on π0.5. That's the bottleneck worth fixing.

[Amish Sethi✈️CoRL @amish_sethi04]

Haha, we had the same exactly thought! You can easily tape any piezoresistive tactile array onto the fingers—we’re using FlexiTac V2 here. Easy to attach and connect to the Raspberry Pi. We are currently exploring training policies that combine vision and gripper state with tactile feedback!

[Amish Sethi @amish_sethi04]

Can collecting robot data be as easy as 𝗽𝗿𝗲𝘀𝘀𝗶𝗻𝗴 𝗿𝗲𝗰𝗼𝗿𝗱?

Introducing UMI-Live, a 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 bimanual UMI we made that takes you from handheld demonstrations all the way to a trained robot policy.

It's built to be easy to use. Each gripper has its own camera, LiDAR, compute, and battery, so there are no base stations to mount and nothing to calibrate in the scene. You press one button and both hands start recording.

While you keep collecting, our pipeline processes finished episodes in parallel and turns them into LeRobot datasets, automatically uploaded on Huggingface. VLAs fine-tuned mostly on this data succeed in 𝟭𝟬/𝟭𝟬 (π0.5) and 𝟴/𝟭𝟬 (MolmoAct2) trials on a real bimanual task.

We're open-sourcing everything, from the hardware designs and build guides to the code and ~40 hours of data 🧵

[Sam Jung @sammjung_]

More to come soon - our team is always looking for students & researchers to sponsor on open ended data work. Reach out if you're interested!

[Chris Paxton @chris_j_paxton]

Open source UMI solution

[Adrian Lin @spotatkd]

UMI-Live is officially released!

Bimanual, in-the-wild data collection is now as easy as pressing a button.

Check out the full details and open-source release on our website!

https://t.co/007eOa3EZT

[Govind Velamoor @gvelamoor]

We made UMIs with:

- No scene calibration / external tracking hardware
- Onboard sensing, compute, battery, storage
- One button bimanual recording
- Auto cloud upload to LeRobot datasets while you record

Super proud of this work!
https://t.co/L7ntHS944v
https://t.co/EZ138ZEhr8

[Amish Sethi✈️CoRL @amish_sethi04]

𝗛𝘂𝗴𝗲 𝘁𝗵𝗮𝗻𝗸𝘀 𝘁𝗼 𝘁𝗵𝗲 𝘁𝗲𝗮𝗺! Co-first authors @gvelamoor and @spotatkd led this work with @sammjung_, @spencermateega, and @brandonyyang02. Huge thanks to @AfterQuery for sponsoring!

We'll present UMI-Live as a poster at the UMI Arena workshop at #CoRL2026 in Austin on Nov 12. Come say hi!

[Amish Sethi @amish_sethi04]

𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲.

The build guide walks through sourcing, printing, and assembly, and the code covers everything from collection to training.

https://t.co/1cqE0FKHvA

🔧 https://t.co/ga9Vl2erIU
💻 https://t.co/PgZkl67Der
🕹️ https://t.co/jt8PaOvF9z
🧠 https://t.co/REZxT32ZnF
📄 https://t.co/eEbL6BRdFX

[Yilun Du @du_yilun]

How can a robot learn to handle delicate objects from vision?

Excited to share VisualForce, where we learn to infer force from the deformation of a soft gripper. We then train a policy to propose actions together with their predicted forces, allowing us to search for actions that apply the right amount of force.

This enables the robot to handle berries without crushing them and insert plugs without slipping, without force or tactile sensors at deployment.

https://t.co/YyNVsb4Y2g

[Haonan Chen @ CoRL @HaonanChen_]

Can a robot see how hard it is squeezing without force sensors?

VisualForce uses gripper deformation to learn force-aware manipulation—from handling berries to inserting plugs—without force or tactile sensors at deployment.

https://t.co/zrzQNYFSQb

1/4

[Haldun Balim @haldunbalim1]

What if embodied agents could learn to plan instead of just learning how to act? 🧠🤖

Introducing Generative Trajectory Planning (GTP)! 🚀

Instead of learning policies through RL, GTP imagines possible futures, plans ahead, and learns to plan better through experience.