Mark Van der Merwe

PhD @UMRobotics | Previously: Intern @NASAJPL, Intern @Mila_Quebec, Undergraduate @UUtah

Computer Vision, Robotics, and Machine/Deep Learning.

[Youngsun Wi @WiYoungsun]

Dexterous hands vary widely—so do tactile modalities. 🖐️🌈
Our vision on tactile human-to-robot transfer:
🔓 Not tied to specific hardware
♻️ Reuse human tactile demos across embodiments

Presenting TactAlign, a cross-sensor tactile alignment for cross-embodiment policy transfer.

[Michigan Robotics @UMRobotics]

Earlier this year, @NimaFazeli7 recieved a @NSF CAREER Award.

We took the chance to catch up with Professor Fazeli to learn a bit more about the fascinating research on "intelligent and dexterous robots that seamlessly integrate vision and touch.”

[Luís Marques @TheLuisMarques]

Is dynamics model mismatch breaking your robotic safety guarantees? We used Conformal Prediction to construct probabilistically safe trajectories given approximate Gaussian dynamics models.

Learn more at https://t.co/ihpAxlXwnu

Presented @wafr_conf & @michigan_AI's Symposium

[Boyang Wang @BoyangWang7]

Our This&That: Language-Gesture Controlled Video Generation for Robot Planning testing code of Video Diffusion Model is released at github:
https://t.co/I7QigcCzxe
The rest will be coming soon (working hard to organizing the code😆)

[Boyang Wang @BoyangWang7]

Verry happy to share our new paper, This&That, an dynamic robot video generation model with language and simple gestures conditioning! Moreover, we also propose Diffusion Video to Action (DiVA) model to transfer generated videos to robot actions in the rollout environment.

[Jeong Joon Park @jjpark3D]

Imagine controlling robots with simple gestures! We've developed a system that lets you point at objects and tell robots to 'move this' or 'close that,' with language-gesture-controlled video generation! Check out our project dubbed "this&that": https://t.co/9eY0eWte4p

[Zixuan Huang @ZixuanHuang15]

MPC is powerful but doesn’t work well for long-horizon tasks without reference trajectories, which are usually expensive to compute online

Introducing Subgoal Diffuser to generate subgoals at appropriate temporal resolution dynamically to guide MPC

https://t.co/5yuaxdju8C

🧵↓

[ARM Lab at University of Michigan @umicharmlab]

Zixuan Huang et. al's new #ICRA2024 paper uses diffusion to guide MPC for complex manipulation tasks! Subgoal-Diffuser breaks the task down into reachable subgoals to guide MPC, thus avoiding local minima. Paper: https://t.co/DwkEDHtJue Video: https://t.co/D4o2DG6skI

[Peter Mitrano @PeterMitrano]

I successfully defended my PhD! Thank you to my advisor Dmitry Berenson, and my lab mates in the @umicharmlab ! 🎓🔬🤖

If you're interested in hiring me to work on learning & planning for robotic manipulation, send me a message!

[Youngsun Wi @WiYoungsun]

How can a 🤖 plan and control tools (e.g., 🧽 🧹) for contact-rich tasks given visual-language inputs? CALAMARI 🦑 shows how we can handle this problem in a very generalizable and data-efficient way via a spatial-action map representation! (1/n)
@corl_conf

[ARM Lab at University of Michigan @umicharmlab]

📢A new way to look at motion planning as online learning in Marco Faroni's RA-L paper! *Key idea*: Bias sampling in a Kinodynamic RRT via a non-stationary multi-armed bandit, where arms are clusters of transitions.
Paper: https://t.co/lv4NZMeWZb . Video: https://t.co/4ngSpz5cPI

[Mark Van der Merwe @MarkVanderMerwe]

Check out our work on detecting contact between a deforming tool and its (unknown) environment! #RSS2023

[Nima Fazeli @NimaFazeli7]

Home robots need to learn how to use our compliant tools (e.g., sponges). @MarkVanderMerwe's #RSS2023 paper enables simultaneous deformation and contact estimation via implicit representations, paving the way for next-gen dexterous tool-use @umicharmlab @UMRobotics @WiYoungsun

[Nima Fazeli @NimaFazeli7]

Robots can slide objects on flat surfaces when they are too big/heavy to lift. But what if they cannot push on the side of the object? Xili Yi shows how robots can use top contact to certifiably push the object to any desired configuration #RSS2023 https://t.co/BsXQ5VncjV

[Nima Fazeli @NimaFazeli7]

Close your eyes and pick up two objects, one in each hand. Can you guess their poses just from poking them against each other? Our robots can! Check out MultisSCOPE, Andrea Sipos' paper accepted at RSS 2023: https://t.co/gYGp7fb6MW #RSS2023 @UMRobotics @UMengineering

[ARM Lab at University of Michigan @umicharmlab]

Tactile pose estimation is difficult when we have only a few contacts on the object. Johnson Zhong's CHSEL uses a Quality-Diversity alg. to produce diverse plausible poses from contact and free space info, outperforming previous work. @NimaFazeli7 #RSS2023 https://t.co/BVlnPQP1f0

[Peter Mitrano @PeterMitrano]

I'll be presenting our paper on FOCUS tomorrow in the morning #ICRA2023 Come see how a clever tweak to the fine tuning process can improve data efficiency for sim2real transfer!

[ARM Lab at University of Michigan @umicharmlab]

@MarkVanderMerwe presenting his work at #CoRL2022! Way to go Mark!

[Nima Fazeli @NimaFazeli7]

📢 How can robots use compliant tools in contact-rich tasks without explicitly modeling their complex mechanics? @MarkVanderMerwe shows a novel multimodal contact-centric approach #CoRL2022! Paper: https://t.co/9PyzoJAUK1 & Page: https://t.co/vOnnk54iB3 @umicharmlab @UMRobotics

[Nima Fazeli @NimaFazeli7]

📢 How can we enable robots to dexterously manipulate tools with high resolution and high deformation tactile sensors? Check out our paper https://t.co/cZTQY1x065 at CoRL 2022! Project Page: https://t.co/oyHbhXqXDE @UMRobotics @umicharmlab

[Nima Fazeli @NimaFazeli7]

📢 Interested in learning to use sight and touch to model deformable objects? Check out our paper VIRDO++ https://t.co/AouUmVSlrk #CoRL2022 to see how implicit representations seamlessly integrate multimodal sensing in the real-world. @WiYoungsun @andyzengtweets @peteflorence