Dexterous hands vary widely—so do tactile modalities. 🖐️🌈
Our vision on tactile human-to-robot transfer:
🔓 Not tied to specific hardware
♻️ Reuse human tactile demos across embodiments
Presenting TactAlign, a cross-sensor tactile alignment for cross-embodiment policy transfer.
Earlier this year, @NimaFazeli7 recieved a @NSF CAREER Award.
We took the chance to catch up with Professor Fazeli to learn a bit more about the fascinating research on "intelligent and dexterous robots that seamlessly integrate vision and touch.”
Is dynamics model mismatch breaking your robotic safety guarantees? We used Conformal Prediction to construct probabilistically safe trajectories given approximate Gaussian dynamics models.
Learn more at https://t.co/ihpAxlXwnu
Presented @wafr_conf & @michigan_AI's Symposium
Our This&That: Language-Gesture Controlled Video Generation for Robot Planning testing code of Video Diffusion Model is released at github:
https://t.co/I7QigcCzxe
The rest will be coming soon (working hard to organizing the code😆)
Verry happy to share our new paper, This&That, an dynamic robot video generation model with language and simple gestures conditioning! Moreover, we also propose Diffusion Video to Action (DiVA) model to transfer generated videos to robot actions in the rollout environment.
Imagine controlling robots with simple gestures! We've developed a system that lets you point at objects and tell robots to 'move this' or 'close that,' with language-gesture-controlled video generation! Check out our project dubbed "this&that": https://t.co/9eY0eWte4p
MPC is powerful but doesn’t work well for long-horizon tasks without reference trajectories, which are usually expensive to compute online
Introducing Subgoal Diffuser to generate subgoals at appropriate temporal resolution dynamically to guide MPC
https://t.co/5yuaxdju8C
🧵↓
Zixuan Huang et. al's new #ICRA2024 paper uses diffusion to guide MPC for complex manipulation tasks! Subgoal-Diffuser breaks the task down into reachable subgoals to guide MPC, thus avoiding local minima. Paper: https://t.co/DwkEDHtJue Video: https://t.co/D4o2DG6skI
I successfully defended my PhD! Thank you to my advisor Dmitry Berenson, and my lab mates in the @umicharmlab ! 🎓🔬🤖
If you're interested in hiring me to work on learning & planning for robotic manipulation, send me a message!
How can a 🤖 plan and control tools (e.g., 🧽 🧹) for contact-rich tasks given visual-language inputs? CALAMARI 🦑 shows how we can handle this problem in a very generalizable and data-efficient way via a spatial-action map representation! (1/n)
@corl_conf
📢A new way to look at motion planning as online learning in Marco Faroni's RA-L paper! *Key idea*: Bias sampling in a Kinodynamic RRT via a non-stationary multi-armed bandit, where arms are clusters of transitions.
Paper: https://t.co/lv4NZMeWZb . Video: https://t.co/4ngSpz5cPI
Home robots need to learn how to use our compliant tools (e.g., sponges). @MarkVanderMerwe's #RSS2023 paper enables simultaneous deformation and contact estimation via implicit representations, paving the way for next-gen dexterous tool-use @umicharmlab@UMRobotics@WiYoungsun
Robots can slide objects on flat surfaces when they are too big/heavy to lift. But what if they cannot push on the side of the object? Xili Yi shows how robots can use top contact to certifiably push the object to any desired configuration #RSS2023 https://t.co/BsXQ5VncjV
Close your eyes and pick up two objects, one in each hand. Can you guess their poses just from poking them against each other? Our robots can! Check out MultisSCOPE, Andrea Sipos' paper accepted at RSS 2023: https://t.co/gYGp7fb6MW #RSS2023@UMRobotics@UMengineering
Tactile pose estimation is difficult when we have only a few contacts on the object. Johnson Zhong's CHSEL uses a Quality-Diversity alg. to produce diverse plausible poses from contact and free space info, outperforming previous work. @NimaFazeli7#RSS2023 https://t.co/BVlnPQP1f0
I'll be presenting our paper on FOCUS tomorrow in the morning #ICRA2023 Come see how a clever tweak to the fine tuning process can improve data efficiency for sim2real transfer!
📢 How can robots use compliant tools in contact-rich tasks without explicitly modeling their complex mechanics? @MarkVanderMerwe shows a novel multimodal contact-centric approach #CoRL2022! Paper: https://t.co/9PyzoJAUK1 & Page: https://t.co/vOnnk54iB3 @umicharmlab@UMRobotics
📢 How can we enable robots to dexterously manipulate tools with high resolution and high deformation tactile sensors? Check out our paper https://t.co/cZTQY1x065 at CoRL 2022! Project Page: https://t.co/oyHbhXqXDE @UMRobotics@umicharmlab
📢 Interested in learning to use sight and touch to model deformable objects? Check out our paper VIRDO++ https://t.co/AouUmVSlrk #CoRL2022 to see how implicit representations seamlessly integrate multimodal sensing in the real-world. @WiYoungsun @andyzengtweets @peteflorence