locoxsoco

PhD Student in CS at @la_UPC researching RL for Humanoid Character Control 🤸‍♀️

[Pieter Abbeel @pabbeel]

Game changer for humanoid full-body control training. Now it's superfast (and cheap, just one GPU) to train a single policy on *all* the motions. How come:
(1) World model much faster than regular sim
(2) Backprop through time collects way more signal than policy gradients

[locoxsoco @locoxsoco_2111]

Love to see locomotion in more and more complex scenarios

[Mateo Guaman Castro @mateoguaman]

Our manipulation results were mind-blowing the first time I saw them, but earlier on, my favorite result was this locomotion run. Many great people have done parkour-type skills with locomotion already, but as far as I am aware, most, if not all, of these works trained separate policies for each terrain, and then distilled into a single policy. They also typically contained tons of hand-engineered rewards and curriculums.

A very early motivation for SGS was to simply figure out if we could get similar results but get rid of as much of this hand-engineering as possible, and we managed to achieve that! This video shows a continuous 1-min run in much harder terrains than ever before, all solved by the same policy.

[Junzhe (JJ) He @JayHe748646]

Will release something interesting in the coming months.

[locoxsoco @locoxsoco_2111]

@jinrui82 Thanks for the insight!

[Chen Tessler @ChenTessler]

At @nvidia, we built ProtoMotions to help us, and researchers world-wide, innovate quickly without compromising on applicability.

We're proud to announce ProtoMotions3 -- our biggest release yet!
🧵👇

[Jason Peng @xbpeng4]

MimicKit now supports #IsaacLab! After many years with IsaacGym, it's time to upgrade. MimicKit has a simple Engine API that allows you to easily swap between different simulator backends.

Which simulator would you like to see next?

[Jason Peng @xbpeng4]

Implementing motion imitation methods involves lots of nuisances. Not many codebases get all the details right. So, we're excited to release MimicKit!
https://t.co/7enUVUkc3h
A framework with high quality implementations of our methods: DeepMimic, AMP, ASE, ADD, and more to come!

[Zhengyi “Zen” Luo @zhengyiluo]

This has to be the best real-time whole-body teleoperation I have ever seen. Super-responsive, low-latency, and off-gantry.

[Zhengyi “Zen” Luo @zhengyiluo]

And once you start using Unitree, it's going to be very resource-intensive to change to other humanoid platforms, as APIs/Codebases will be overfitted to one. Huge first-mover advantage. Other companies will need to deliver significantly more value to convince people to switch.

[locoxsoco @locoxsoco_2111]

@ChenTessler @KyleMorgenstein Big fan of ProtoMotions! Not only saved me tons of hours migrating from IsaacGym to IsaacLab, but the modular structure lets me run many tests by updating only what's relevant. Plus, the vanilla-style RL algorithms code make it super easy to plug in novel techniques. Huge thanks!

[locoxsoco @locoxsoco_2111]

@sang_yun_lee The paper I had in mind was this: https://t.co/VZVaCXDTcu

[locoxsoco @locoxsoco_2111]

@sang_yun_lee I don't have THE answer to that, but I would say learning to recognize patterns. It reminds me how vision-language-action (VLA) frameworks uses image models (like DINO) to process images into a latent space that other models can then use. What do you think?

[locoxsoco @locoxsoco_2111]

@sang_yun_lee Oh, I meant that the way the brain is studied suggests it has different learning objectives depending on the region, and that we should not consider our modelling bad since, compared to a brain, we are just training regions (for now).

[locoxsoco @locoxsoco_2111]

Reading of the weekend! Looking forward to representatives latent spaces:

FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning

[Chenhao Li @breadli428]

✨ICLR 2024 Spotlight

Still hand-labeling phases or tuning GANs with your trajectories?

We propose FLD, a self-supervised representation method that extracts spatial-temporal relationships in high-dimensional trajectories. RL policies informed by FLD show extended generality.🧵

[locoxsoco @locoxsoco_2111]

@ZhaoMandi If the goal is to have all keypoints as close to the object, what do you think about using ‘rew = torch.min(exp(-beta * dist))’ to try minimizing all distances? Would you recommend it depending on the task or prefer using the mean for most cases?

[Mandi Zhao @ZhaoMandi]

One amusing takeaway from doing RL in these massively parallelized sim environments is that reward engineering matters more so than ever.

A small detail in reward function could make a huge difference: with 10k+ parallel threads to explore in, the policy Will exploit any caveat and find a shortcut to high rewards and not going with the desired behavior.

The two videos below are only one line difference in implementation: left computes the average distance between the 4 fingertip keypoints and the object, and right is adding the palm as another keypoint in the same function, which quickly leads to the grasping motion and lifts up the object.

[locoxsoco @locoxsoco_2111]

Learning to move with hands! ✋🤚

Can’t wait to share my work: PHA: Part-wise Heterogeneous Agents with Reusable Policy Priors for Physics-Based Motion Synthesis at #SCA2025

Video and paper coming soon!

[locoxsoco @locoxsoco_2111]

Reading for the weekend! My agents may need more than manual reward terms so this looks promising.

📄 AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning (https://t.co/6KZ8MnD5zo)

https://t.co/c57q8zjgyK