Govind Velamoor

Student @yale

[Govind Velamoor @gvelamoor]

@kdpisda @amish_sethi04 We interpolate!

[Chris Paxton @chris_j_paxton]

Open source UMI solution

[Govind Velamoor @gvelamoor]

@srigopal @amish_sethi04 We sync the clocks of the UMIs and timestamp camera frames against those clocks. We measure gripper opening from the ArUco tags in the images (no other sensor for width). Then we align the two recordings by timestamp.

[Govind Velamoor @gvelamoor]

We made UMIs with:

- No scene calibration / external tracking hardware
- Onboard sensing, compute, battery, storage
- One button bimanual recording
- Auto cloud upload to LeRobot datasets while you record

Super proud of this work!
https://t.co/L7ntHS944v
https://t.co/EZ138ZEhr8

[Amish Sethi @amish_sethi04]

Can collecting robot data be as easy as 𝗽𝗿𝗲𝘀𝘀���𝗻𝗴 𝗿𝗲𝗰𝗼𝗿𝗱?

Introducing UMI-Live, a 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 bimanual UMI we made that takes you from handheld demonstrations all the way to a trained robot policy.

It's built to be easy to use. Each gripper has its own camera, LiDAR, compute, and battery, so there are no base stations to mount and nothing to calibrate in the scene. You press one button and both hands start recording.

While you keep collecting, our pipeline processes finished episodes in parallel and turns them into LeRobot datasets, automatically uploaded on Huggingface. VLAs fine-tuned mostly on this data succeed in 𝟭𝟬/𝟭𝟬 (π0.5) and 𝟴/𝟭𝟬 (MolmoAct2) trials on a real bimanual task.

We're open-sourcing everything, from the hardware designs and build guides to the code and ~40 hours of data 🧵

[Amish Sethi @amish_sethi04]

Can collecting robot data be as easy as 𝗽𝗿𝗲𝘀𝘀𝗶𝗻𝗴 𝗿𝗲𝗰𝗼𝗿𝗱?

Introducing UMI-Live, a 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 bimanual UMI we made that takes you from handheld demonstrations all the way to a trained robot policy.

It's built to be easy to use. Each gripper has its own camera, LiDAR, compute, and battery, so there are no base stations to mount and nothing to calibrate in the scene. You press one button and both hands start recording.

While you keep collecting, our pipeline processes finished episodes in parallel and turns them into LeRobot datasets, automatically uploaded on Huggingface. VLAs fine-tuned mostly on this data succeed in 𝟭𝟬/𝟭𝟬 (π0.5) and 𝟴/𝟭𝟬 (MolmoAct2) trials on a real bimanual task.

We're open-sourcing everything, from the hardware designs and build guides to the code and ~40 hours of data 🧵