GPT-6 Astra is extremely good at creating digital twins 🤯!
I fed it a few real-world episodes and it was able to create a near exact replica in Mujoco + Blender
One step closer to closing the real2sim gap
GPT-6 Astra is extremely good at creating digital twins 🤯!
I fed it a few real-world episodes and it was able to create a near exact replica in Mujoco + Blender
One step closer to closing the real2sim gap
You can collect robot data without touching a robot 😮.
...using UMI (Universal Manipulation Interface).
I implemented the original version of UMI ... with a few modifications
I modified it slightly to account for:
- shorter finger gripper length
- moving the camera further forward
Using a UMI device has huge advantages for data collection:
- much faster than teleoperating a robot
- low cost and easy to use
- natural human motions
- no robot required during data collection
But it also comes with some real challenges:
- calibration and localization
- custom code to convert human demonstrations into something a robot can actually learn from
I’m going to share a few lessons from building the device, collecting UMI data, and training policies on it.
Follow along if you’re interested.
@DominiqueCAPaul I was watching their videos on YouTube three weeks back and noticed the same thing.
I assume they're using it for localization, but it could just be as a reference ID