Prox (@try_prox) builds digital co-workers for 3PLs and fulfillment centers - companies that store, pack, and ship for brands.
They automate the back-office operations layer that burns billions of labor hours across the industry.
1/4 We recreated a $200k teleoperation setup in VR for just ~$2k. Now we can collect more dextrous manipulation data in a single day (40 hrs/day) than any existing open dataset has ever collected.
4/4 We currently have 5 Apple Vision Pros, which means our throughput is roughly 40 hours of teleoperation data per day. This means we can collect more data per day than any existing open dataset in this category has ever collected.
Here's how we did it: https://t.co/BJqZXVYvwY
3/4 Not having those kinds of resources, we built an interface to teleoperate bimanual Shadow Hands in Apple Vision Pro. Robotics always starts with failure, so we spent weeks failing over and over again until we got the setup running smoothly at 30Hz.
1/ I've created a detailed beginner-friendly guide on how to assemble a low-cost dextrous robotic hand for ~$450 using parts available entirely on Amazon. The hand utilizes a tendon-drive design with 17 DoF and can be easily teleoperated via webcam.
5/ App Store: https://t.co/E3vBhWzoE1
Project website: https://t.co/KQ6kvcXOmu
The DART project was developed at the Improbable AI Lab at MIT CSAIL, under @younghyo_park and @pulkitology.
DM me if you want to try in-person teleoperation in Vision Pro. I’ll be in SF this summer
1/Robotics foundation models need internet-scale data to train, and we bet simulation datasets will help reach this scale. It's time to build tools enabling this. So, we're releasing first public app for robot data collection in Vision Pro and a platform to curate datasets
4/ https://t.co/2u7Yggd0D1 runs beautifully on mobile devices as well, since it is powered by MuJoCo compiled to WebAssembly. And this isn’t just a video recording!! It’s the actual physics engine running in your browser, replaying the demonstrations in real time.