Rui Chen

Ph.D. candidate @CMU_Robotics with @ICL_at_CMU. Constrained Control/Optimization/Learning. @SCSatCMU @UMich

[Rui Chen @RuiChen_Rob]

Stay tuned!

[Yifan Sun @YifanSun98]

[1/4] 🌟Sneak Peek: SPARK in Action! 🦾
Previewing Safe Protective & Assistive Robot Kit (SPARK)—a modular toolbox designed to enhance safety in humanoid autonomy and teleoperation.

Safety isn't just a feature—it's the foundation for humanoids to truly integrate into human life. SPARK filters risky actions, ensuring humanoids can achieve their objectives securely across tasks—from lab experiments to real-world deployments. With SPARK, you can innovate fearlessly, knowing safety is always guaranteed.

Powered by Safe Set Algorithm (SSA), SPARK is built to:
✅ Configure safety behaviors with ease
⚖️ Balance safety and performance
🤖 Integrate with Unitree G1 + Apple Vision Pro
🔧 Support customization for other systems

Stay tuned for the full release in a few weeks 🚀

Please see our website for the paper and more details!
🌐 Website: https://t.co/1e879HhH4C

@ICL_at_CMU @CMU_Robotics @CarnegieMellon @UnitreeRobotics

#Robotics #HumanoidSafety #AIInnovation

[Changyi Lin @changyi_lin1]

Introducing LocoTouch:
Quadrupedal robots equipped with tactile sensing can now transport unsecured objects — no mounts, no straps.
The tactile policy transfers zero-shot from sim to real.

Core Task-Agnostic Features:
1. High-fidelity contact simulation for distributed tactile sensors
2. Adaptive gait reward for symmetric, frequency-adaptive locomotion

Website: https://t.co/WiycKk7NVO
Paper: https://t.co/NqRKxBcNZi

[Intelligent Control Lab @ICL_at_CMU]

🎉 Congrats to Prof Changliu Liu @ChangliuL on IEEE RAS Early Academic Career Award in Robotics and Automation "for significant advancements in intelligent robot control and learning that enables safe and efficient human-robot collaborations"! @CMU_Robotics #ICRA2025

[Hongyi Chen @chen_hongyi_]

[1/7] Teaching dexterous robot hands to perform functional grasps usually needs hours of teleoperation, manual labeling, or pre-scanning object meshes.

Not anymore.
🔥We are excited to introduce Web2Grasp that learns functional multi-finger grasps straight from web images of human hand-object interactions (HOI).
No human demos. No object scans. Just web images.
👉https://t.co/lOcWgMOpu5
@CMU_Robotics @CarnegieMellon

[Peide Huang @peide_huang]

🚀 New Research on Human-Robot Interaction! 🤖

How can humanoid robots communicate beyond words? Our framework, EMOTION, leverages Large Language Models (LLMs) to dynamically generate expressive gestures, enhancing non-verbal communication in robots.

🤯 Our experiments show that EMOTION can generate various expressive gestures from only TWO examples and match human-generated gestures in understandability & naturalness!

🔍 What’s inside?
✅ LLM-powered motion generation
✅ Human feedback to refine gestures (EMOTION++)
✅ 10 expressive gestures generated and evaluated (thumbs-up, stop, jazz-hands & more!)

📜 Read the full paper: https://t.co/UOYItwsEe0
🎬 Watch the video: https://t.co/O2VkbezW2o

Let’s bring robots closer to human-like interactions! What gestures would you like to see next? 👇

Huge kudos to the amazing team at Apple that made this work @Yuhan_Hu_, Nataliya Nechyporenko, @talking_kim, @waltertalbott, @jian_zhang_.

#Robotics #HRI #LLMs #HumanRobotInteraction #GestureGeneration #SocialRobots

[Rui Chen @RuiChen_Rob]

@JiachengZhu_ML @AIatMeta Nice!! Congrats!!

[Tairan He @TairanHe99]

🚀 Can we make a humanoid move like Cristiano Ronaldo, LeBron James and Kobe Byrant?

YES!

🤖 Introducing ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

Website: https://t.co/XQga7tIfdw
Code: https://t.co/NpEeJtVxpp

[Intelligent Control Lab @ICL_at_CMU]

🚀 Our work is featured in the latest @ARM_Robotics article on the Digital Framework for Sustainment & Maintenance! 🎯🔧

Check out how our innovation is shaping the future of digital sustainment: 🔗 https://t.co/BR57O2VwZe

#AI #Robotics #ManufacturingInnovation

[Feihan Li @Li_liangzhii]

🤖Can highly nonlinear legged robots be controlled by simple linear controller?
🌟Yes—with Incremental Koopman!

Our Incremental Koopman approach transforms nonlinear dynamics into linear models, enabling powerful, proven control techniques. By iteratively expanding datasets and latent spaces, it ensures precise convergence to true system dynamics with monotonic error reduction.

✅This project delivers scalable, high-performance control across terrains and legged robots like Unitree & ANYmal.
🦾A solution for linearized control in locomotion!

Please see our website for the paper and more details:
🌐Website: https://t.co/vUH78l1nZ7

@ICL_at_CMU @CMU_Robotics @CarnegieMellon

#Robotics #Humanoid #AIInnovation

[Hao-Shu Fang @haoshu_fang]

A good hand can push intelligence development. Introducing Eyesight Hand, equipped with full-hand high-res tactile sensors and proprioceptive actuators. It is compliant, agile, and powerful. Good tactile sensing makes learning more efficient and robust. Shout out to Branden!

[Rui Chen @RuiChen_Rob]

You're correct. The feature isn't active for everything; users decide what to avoid (e.g., people nearby) and what to touch (e.g, things to grab). Teleoperators can use it to prevent collisions from mis-operations or unintended commands. Think of it like a driver-assistance system—engaging only when needed.

[Rui Chen @RuiChen_Rob]

An Apple Vision Pro handles most of the sensing.

When it’s carrying a glass of liquid, one can add constraints to enforce that the hand always holds the glass upright, and perhaps not moving too aggressively. Conditions like those can be added with a bit of modification.

Yes features complicate things. Totally agree

[Hongyi Chen @chen_hongyi_]

#CoRL2024 accepted!🌈

Our work KOROL developed a linear dynamics model using object features that capture key information for robotic manipulation, outperforming models that rely on GT object states.

Code: https://t.co/3WWJZGsH1S

[Hanjiang Hu @huhanjiang]

How can we solve MPC with neural dynamics efficiently but still with safety guarantees?

Check out our #L4DC2024 poster "Real-Time Safe Control of Neural Network Dynamic Models with Sound Approximation" with Prof. Jianglin Lan and Prof. Changliu Liu @ChangliuL @ICL_at_CMU

[Rui Chen @RuiChen_Rob]

Annoyed by trying out control designs one by one for each new task?

Just ask Meta-Control!

[Tianhao Wei @wei_tianhao]

🤔 How can a robot accomplish heterogeneous tasks, with precision and guarantees?

🚀 Introducing Meta-Control: Automating model-based control synthesis for heterogeneous skills with LLM!

https://t.co/zeN9aoiqG5

[Kevin Zakka @kevin_zakka]

Center-of-mass regulation + feet/hands on the G1 humanoid.

[Animesh Garg @animesh_garg]

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU.

However, the methods use 0-th order gradients, and are often not the best optimizers.

Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳

tl;dr: Faster, better RL than PPO in continuous control 💪
https://t.co/fmFoMMWFAZ

The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well?
This has been a driving question in a series of our works.

We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic
https://t.co/pTDtrA2gys
Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients.

SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim.
This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids

Yet, given enough compute PPO often caught up.
Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix.
https://t.co/fmFoMMWFAZ

However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error.
1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact.

We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy.
Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact.

AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO.

This work is led by @imgeorgiev alongside @krishpopdesu, @xujie7979, @eric_heiden and ample assistance from warp team at @NVIDIARobotics (@milesmacklin)

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

Attention as an RNN

abs: https://t.co/Ewb4WafnPA

"attention can be viewed as an RNN with the special ability to compute its many-to-one RNN output efficiently"

Proposes Aaren, a new module that can be trained in parallel (like Transformers) but also be efficiently updated at inference time, thereby requiring only constant memory (like RNNs).

[Rerun @rerundotio]

Go2 in Rerun: visualizing @UnitreeRobotics Go2 quadruped using Rerun’s ROS 2 bridge PoC.

This example visualizes the Go2 robot, its onboard sensor data, and how it sees the environment. It’s built using Rerun’s new proof-of-concept C++ ROS 2 bridge, which currently supports a limited set of message types.

Rerun is, to our knowledge, the most performant multimodal visualizer out there and is already used by some of the best teams in the world. We’re continuing our push to make Rerun useful for more Robotics teams and this example and PoC ROS 2 bridge is another step in that direction.

Official ROS 2 and MCAP support will follow, but in the meantime, we hope this example will provide a useful starting point for anyone who wants to use Rerun with their ROS 2 project.

#ROS

#Multimodal

#Robotics

[The Humanoid Hub @TheHumanoidHub]

"Teleoperation fundamentally does not scale." — Jim Fan

Using teleoperators for guiding humanoid robots for data collection is time-consuming and costly. Also, there are practical limits on the scenarios where Optimus can be deployed for data-collection in the real-world.