Our winning setup paired our fine-tuned Dream-1 checkpoint with a VLM planner + CV-based state tracking, and it beat other teams in a surprise 1v1 playoff!
We finetuned Dream-1 on a USB unplugging task using just 50 examples and 500 training steps, and it worked remarkably well on a new embodiment and environment
Entered my first robot hackathon and won!
Big thanks to @viamrobotics and @MongoDB for sponsoring, @RoboticsCtr for hosting, and to my insanely cracked teammate @joeeriqat!
First time entering a hackathon, first time winning a hackathon.
Massive thank you to @MongoDB and @viamrobotics for hosting and especially @anishrav1 for being a goated teammate
Your code. Real robot arms. 48 hours to build.
Robot Hackathon with @MongoDB and @viamrobotics at Robotics Center, Oct 29–30.
Build physical AI applications using real hardware, Viam and MongoDB Atlas.
Event details: https://t.co/AqwM2CTgPI
#PhysicalAI#Robotics
3/ testing a hybrid policy with consistent handoff
on 10 selected sim starts: pure ACT made 2 loops. expert bracelet approach → ACT made 6 clean loops.
is it worth pursuing an end-to-end ACT policy or to keep working on the hybrid? the next task variant moves the tower too
teaching an SO-101 to loop a bracelet over a tiny Eiffel Tower with ACT + @LeRobotHF.
real-robot clip below. the hard part is generalizing to new starting positions.
how would you improve spatial generalization without just collecting more demos?
more in 🧵
2/ more coverage helped, but didn't solve it
271 sim episodes, 100k ACT updates on an L40S:
10/12 on training starts
7/20 on new spatial holdouts
seems like spatial interpolation needs more training data and/or closer starting position clusters