Jeff Ichnowski

[Jeff Ichnowski @jeff_ichnowski]

Modality Forcing is an amazing text-to-RGB(D) recipe developed by Bart Duisterhof and team at @theworldlabs. While we're excited about its robotics applications, its Corgi-Anything demo is what really makes it stand out. #CMUrobotics

[Bardienus Duisterhof @BDuisterhof]

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation!

Text-to-image (T2I) models learn rich representations of the spatial world.

How do we build on this prior for high-quality depth generation?

https://t.co/uJjGHNiDBu

🧵 [1/6]

[Jeff Ichnowski @jeff_ichnowski]

Supercool project on retargeting VR robot manipulation demos to soft hands, with contact forces!

[Uksang Yoo @UksangYoo]

Excited to share SoftAct, a framework for retargeting human manipulation demos to soft robot hands using explicit contact force reasoning! How do you transfer human skill to a hand that looks and moves nothing like yours🐙🖐️? It turns out VR environments can let us capture privileged force interaction demonstrations to help. 🧵1/7

[Jeff Ichnowski @jeff_ichnowski]

Very exciting project led by Hongyi Chen: Robot dexterity from RGB videos.

[Hongyi Chen @chen_hongyi_]

How far can we push dexterous robot manipulation with human video-only supervision and minimal assumptions?
🚫 No teleop. 🚫 No wearables. 🚫 No external sensors. 🚫 No robot demos.
Introducing VIDEOMANIP: 🎥 Just monocular RGB, 🌍 in-the-wild human video → dexterous robot manipulation 🤚[1/6]

[Yu Chen @YuChen97026150]

🤖🦾✍️Why is robot grasping hard? We usually blame contacts, kinematics, geometries, perception, and so on. But what if the object is just being spiteful?

🔥We propose a game-theoretic grasp synthesis method as a two-player game between the robot and an adversarial (spiteful) object.

💡In this formulation we achieve SOTA grasp success rates, without training data - just using optimization tools (Augmented Lagrangian + Iterative Best Response).

📑Arxiv: https://t.co/mH2ocISuHy
🌐Website: https://t.co/HKayRv9yRD

🧵[1/7]

[Jeff Ichnowski @jeff_ichnowski]

Check out Chengyang's fantastic work on robot haircare! (That's robots performing hair styling, not robots with hair--that's future work).

[Chengyang Zhao @alecyzh]

🤖What if a robot could understand hair dynamics well enough to style your hair, just like your favorite barber💈?

🔥Excited to announce DYMO-Hair, a model-based robot hair styling system powered by a generalizable 3D hair dynamics model.

🚀A new step toward robots that can understand and manipulate more complex deformable materials.

New dynamics learning paradigm. Fully synthetic data from a novel lightweight simulator. Zero-shot sim2real transfer.
👉Check it out at: https://t.co/Kx2gyWBAAD
More details below: [1/N]

[Chengyang Zhao @alecyzh]

🤖What if a robot could understand hair dynamics well enough to style your hair, just like your favorite barber💈?

🔥Excited to announce DYMO-Hair, a model-based robot hair styling system powered by a generalizable 3D hair dynamics model.

🚀A new step toward robots that can understand and manipulate more complex deformable materials.

New dynamics learning paradigm. Fully synthetic data from a novel lightweight simulator. Zero-shot sim2real transfer.
👉Check it out at: https://t.co/Kx2gyWBAAD
More details below: [1/N]

[Hongyi Chen @chen_hongyi_]

[1/7] Teaching dexterous robot hands to perform functional grasps usually needs hours of teleoperation, manual labeling, or pre-scanning object meshes.

Not anymore.
🔥We are excited to introduce Web2Grasp that learns functional multi-finger grasps straight from web images of human hand-object interactions (HOI).
No human demos. No object scans. Just web images.
👉https://t.co/lOcWgMOpu5
@CMU_Robotics @CarnegieMellon

[Yunchao Yao @YaoYunchao]

Can soft robots rapidly spin pens like humans?🤔 We’ve shown that soft robot hands can master the dynamic tasks of pen spinning—no hours of GPU training or complex sim-to-real needed! Check out https://t.co/DRnsbusWT3. 🤖✍️ @CMU_Robotics.
1/5🧵

[Uksang Yoo @UksangYoo]

Can robots make pottery🍵? Throwing a pot is a complex manipulation task of continuously deforming clay. We will present RoPotter, a robot system that uses structural priors to learn from demonstrations and make pottery @HumanoidsConf @CMU_Robotics
👇https://t.co/kOvQpkUyY3 1/8🧵

[Hongyi Chen @chen_hongyi_]

#CoRL2024 accepted!🌈

Our work KOROL developed a linear dynamics model using object features that capture key information for robotic manipulation, outperforming models that rely on GT object states.

Code: https://t.co/3WWJZGsH1S

[Bardienus Duisterhof @BDuisterhof]

Dense tracking of deformable objects can unlock applications in robotics, gen-AI and AR. We present DeformGS (previously MD-Splatting) and release the code and data.  Join us at #WAFR where we will present new real-world results!
👇https://t.co/6m6MODxxAi 1/9🧵

[Ken Goldberg @Ken_Goldberg]

Dive into #CloudRobotics at the hands-on #ICRA24 Tutorial on 17 May! 🤖💡
🎓 Led by academia & industry leaders
🔍 Explore cutting-edge @ROSorg tools
🛠️ Get hands-on with #FogROS2 & @CloudGrippe
From #UCBerkeley, #CMU, #KTH, @ieee_ras_icra
https://t.co/XfpgKJA6ni

[Bardienus Duisterhof @BDuisterhof]

Deformable objects are common in household, industrial and healthcare settings. Tracking them would unlock many applications in robotics, gen-AI, and AR.

How? Check out MD-Splatting: a method for dense 3D tracking and dynamic novel view synthesis on deformable cloths. 1/6🧵

[Jeff Ichnowski @jeff_ichnowski]

FogROS 2 is from @UC_Berkeley @Autolab and @RISElab led by @jeff_ichnowski w/ @EricKaiyuan, Karthik Dharmarajan, @funmilore, @DanielczukMike, @vmayoralv, Hugo Zhan, Derek Xu, Ramtin Ghassemi, John Kubiatowicz, Ion Stoica, @mejoeyg, and @Ken_Goldberg. https://t.co/dypIZxRWlU (7/7)

[Jeff Ichnowski @jeff_ichnowski]

The latest Robot Operating System (@ROSorg) now includes FogROS 2, an open-source platform for offloading computation of deep learning, motion planning, grasp planning, and map building to cloud systems such as @AWScloud. (1/7)

[Jeff Ichnowski @jeff_ichnowski]

In benchmarks, FogROS 2 accelerates SLAM (https://t.co/wQScjOrgNk) by 2x, Dex-Net (https://t.co/YQ1hAoP1a6) on a GPU by 11.7x, and Motion Planning (https://t.co/YFvJPlzsRn) on a 96-core computer by 28x. (6/7)