Jean Oh

roBot Intelligence Group (BIG) at CMU

[Uksang Yoo @UksangYoo]

SoftAct suggests that contact geometry and force distribution are essential for bridging extreme embodiment mismatch that cannot be recovered through kinematic imitation alone. It would've been impossible without some amazing collaborators at @CMU_Robotics and Meta Reality Labs! Website for more details, robot rollouts, and experiments: https://t.co/zUJJPv8xr0

[Uksang Yoo @UksangYoo]

With the retargeted demonstrations, we train behavior cloning policies and achieve zero-shot real-world success rates of 95% on light bulb screwing, 85% on cup pouring, and 70% on box reorienting vs. 30%, 35%, and 10% for kinematic retargeting.

[Uksang Yoo @UksangYoo]

We know soft robot hands 🐙 can do remarkable things like learn dynamic tasks by leveraging stored elastic energy 🖊️and interact safely with people in physical human-robot interaction tasks 💇‍♀️. But teaching them dexterous skills often relied on handcrafted primitives or embodiment-specific demonstrations. Standard kinematic retargeting fails, suggesting we need to use different information from the demonstrations. 🧵2/7

[Uksang Yoo @UksangYoo]

Excited to share SoftAct, a framework for retargeting human manipulation demos to soft robot hands using explicit contact force reasoning! How do you transfer human skill to a hand that looks and moves nothing like yours🐙🖐️? It turns out VR environments can let us capture privileged force interaction demonstrations to help. 🧵1/7

[Chengyang Zhao @alecyzh]

🤖What if a robot could understand hair dynamics well enough to style your hair, just like your favorite barber💈?

🔥Excited to announce DYMO-Hair, a model-based robot hair styling system powered by a generalizable 3D hair dynamics model.

🚀A new step toward robots that can understand and manipulate more complex deformable materials.

New dynamics learning paradigm. Fully synthetic data from a novel lightweight simulator. Zero-shot sim2real transfer.
👉Check it out at: https://t.co/Kx2gyWBAAD
More details below: [1/N]

[Jean Oh @jeanoh]

Due to a number of requests, the submission deadline for the IEEE RA Magazine Special Issue on the Arts and Robotics has been extended to August 15th. The formatting instructions for both technical and non-technical papers can be found on the website: https://t.co/CEIpVKHuDm

[Jean Oh @jeanoh]

We are organizing the first "Robot Fashion Show" at CoRL/Humanoids25 and invite artists, designers, and researchers to propose creative ideas for “dressing up” the robots for safety and aesthetics.

Deadline: July 15, 2025

https://t.co/g8MyQsdVda

#robotFashionShow #designer

[Jean Oh @jeanoh]

We are launching a new platform to build a diverse and friendly multicultural community. Please sign up to join our waitlist or try the Beta version to earn points by posting your photos or explore other cultures in the world!
https://t.co/vKGvlbkCKr

#WorldPeace #WorldCCUB

[Jean Oh @jeanoh]

IEEE RAMagazine welcomes articles for the Special Issue on the Arts and Robotics: https://t.co/hqoN1VWxbe

[Eliot Xing @etaoxing]

@vernon_luk @jeanoh Check out our website, paper, or code to learn more:
website: https://t.co/6fWA7iqcIb
paper: https://t.co/phdyOxSO0V
code: https://t.co/NhVmBhya9A

All of our code, simulation environments & algorithms, is open source and available now:
𝐩𝐢𝐩 𝐢𝐧𝐬𝐭𝐚𝐥𝐥 𝐫𝐞𝐰𝐚𝐫𝐩𝐞𝐝

7/7

[Eliot Xing @etaoxing]

This was a wonderful collaboration over last year with @vernon_luk, @jeanoh. And it would not have been possible without the amazing open source effort by the NVIDIA Warp team. 6/n

[Eliot Xing @etaoxing]

We show that SAPO can learn a range of challenging manipulation and locomotion tasks, re-implemented in Rewarped, that involve interaction between rigid bodies, articulations, and deformables.

SAPO beats model-free RL (PPO, SAC) as well as prior first-order RL (APG, SHAC). 5/n

[Eliot Xing @etaoxing]

SAPO (Soft Analytic Policy Optimization): maximum entropy first-order RL algorithm that uses analytic gradients from diff. sim to train a stochastic actor to maximize expected return and entropy.

Why MaxEnt RL? Smoother optimization landscapes stabilize first-order RL. 4/n

[Eliot Xing @etaoxing]

Rewarped: parallel differentiable multiphysics simulation platform that supports simulating various materials beyond just rigid bodies.

Another simulator? Re-write tasks in Rewarped and scale first-order RL on tasks involving rigid and soft body interaction. 3/n

[Eliot Xing @etaoxing]

Model-free RL is too sample inefficient to scale on slow soft body simulation. Prior first-order RL algorithms work on rigid body locomotion, but fail to solve soft body manipulation. No existing simulator satisfies all: (1) parallel envs (2) differentiable (3) multiphysics. 2/n

[Eliot Xing @etaoxing]

RL is notoriously sample inefficient. How can we scale RL on tasks much slower to simulate than rigid body physics, such as soft bodies?

In our #ICLR2025 spotlight, we introduce both a new first-order RL algorithm, SAPO, and differentiable simulation platform, Rewarped. 1/n

[Jean Oh @jeanoh]

My student, Mehul Agarwal's work was featured in CMU RI News. Together with Mehul's sister, Gauri at MIT Media Lab, we developed a technology to check your identity before using your photograph for genAI. CMU and MIT Join Forces to Combat Deepfake Threats https://t.co/NMJFtVhL9n

[Jean Oh @jeanoh]

Happy Valentine's Day from BIG

[Jean Oh @jeanoh]

Call for Proposals: Arts in Robotics at ICRA’25
(Deadline: Dec 15th)
https://t.co/T72I5jhDDk

[Jean Oh @jeanoh]

RoPotter, our pottery-making robot paper won the Best Oral Presentation Finalist Award at Humanoids 2024 🎉
Uksang Yoo, Adam Hung, Jonathan Francis, Jean Oh, Jeffrey Ichnowski @UksangYoo @jeff_ichnowski
https://t.co/jIHOtGtX8q