Nico Bohlinger

27 | Robotics & RL Research | Intern @ Amazon FAR in SF & PhD student at @ias_tudarmstadt

[Nico Bohlinger @NicoBohlinger]

⚡ One policy, millions of embodiments, over 200 robot models. Can we add yours?

We're building γ₀, a generalist RL policy for motion control trained across millions of randomized embodiments derived from a growing collection of more than 200 robot models.

[Nico Bohlinger @NicoBohlinger]

@_bbelousov How does an additional network reduce lag compared to a PD? Is it predicting ahead, based on inference timings, and therefore reduce lag?

[Chenhao Li @breadli428]

It’s surprising the world model gradient could actually be useful in training policies in imagination, which is opposite to the conclusion drawn from RWM, where such gradients are “sharp” when contact changes. Any insights @qiayuanliao?

And great work as always! Congrats!

[Nico Bohlinger @NicoBohlinger]

@KyleMorgenstein @qiayuanliao I'm surprised by that, for me it works great and also MuJoCo Warp does not seem different in that regard.
But for this paper MJX being good or not in that regard is beyond the point

[Nico Bohlinger @NicoBohlinger]

@erwincoumans @KyleMorgenstein @qiayuanliao I was not aware, looks great.
I totally believe that a WM will have smoother gradients, still I would have liked to see a comparison for their method. But I'm sure we will see it at some point

[Nico Bohlinger @NicoBohlinger]

@erwincoumans @KyleMorgenstein @qiayuanliao MuJoCo Warp is currently not differentiable so SHAC-like algorithms can't use it: https://t.co/cCgma2usK4

[Nico Bohlinger @NicoBohlinger]

Hmm, they are not even doing manipulation in that paper, MJX is pretty fine here. At least to actually have a comparison with a differentiable sim. SHAC and others always use it for a reason so a comparison would help to see how much of the gain is actually the WM.

For non differentiable training you should use MJX with the warp backend. This is always faster than the mjlab style of warp with torch.

[Nico Bohlinger @NicoBohlinger]

@grantg07 I think there is a lot of "co-design" or let's say alignment happening in their actuators and encoders. On the legs probably as well but I think the actuators and their encoders are even more critical for good sim2real

[Nico Bohlinger @NicoBohlinger]

@anishhdiwan added six Imitation Learning and Inverse RL algorithms to RL-X: GAIL, AIRL, AMP, NEAR, TRIRL (with or without differentiable trust region layers).
Implemented in Jax and PyTorch, and fully jitted for maximum speed 🚀
Feel free to check them out: https://t.co/6OEyLqlhWY

[Nico Bohlinger @NicoBohlinger]

Our γ₀ team keeps growing! :D

Thanks to everyone contributing robots and working with us on deployment. Really exciting to see this coming together across so many labs!
278 robots now, under water, in the air and much more coming!

You have more quirky robot models lying around? You want to join us on deployment on new robots? Get in touch by tomorrow, October 15:
[email protected]

[Nico Bohlinger @NicoBohlinger]

⚡ One policy, millions of embodiments, over 200 robot models. Can we add yours?

We're building γ₀, a generalist RL policy for motion control trained across millions of randomized embodiments derived from a growing collection of more than 200 robot models.

[Anish Diwan @anishhdiwan]

📢 New @NeurIPSConf paper with @Jan_R_Peters & @OlegArenz :
"A Geometric Perspective on Reward Function Updates in Inverse Reinforcement Learning"

We show that natural-gradient reward updates in inverse RL require no Fisher matrix at all 🧵👇

[Harold @wahoo_harold]

Testing a new throw-in into a direct kick!

[Nico Bohlinger @NicoBohlinger]

@mateoguaman @octi_zhang @rosario_scaling @patrickhyin @abhishekunique7 @gabe_mrgl Yes I'll be there, let's meet up! :)

[Nico Bohlinger @NicoBohlinger]

@mateoguaman @octi_zhang @rosario_scaling @patrickhyin @abhishekunique7 @gabe_mrgl Would be amazing to see that comparison, but I understand how expensive things can be.
But I think you should at least mention / cite his work :)

[Nico Bohlinger @NicoBohlinger]

Cool work for sure!
Although I heavily disagree with grid-based sampling. The more we scale up these environments, i.e. the task configurations, DR, reward terms. All of these things are often continuous and should be attached to a curriculum for the smoothest and best learning experience. This leads to massive grids where 32k or more cells are essentially not covering anything.
Performance based curricula are essential but grid-based is the wrong way imo

[Nico Bohlinger @NicoBohlinger]

@mateoguaman @octi_zhang @rosario_scaling @patrickhyin @abhishekunique7 I would love to see a performance comparison to the box adaptive curriculum from @gabe_mrgl work "Rapid Locomotion via Reinforcement Learning" https://t.co/y8hjaDQz9C

[Manda @Mandarobotics]

Comparing major physics engines on robot tasks, we found stark differences in how Newton, native MuJoCo and Genesis handle deformable objects.

We compared behavior and forces with matched geometry across increasingly complex scenarios. 🧵

[Sharon Lee @sharonal_lee]

Been seeing a lot of Astra for real2sim, but what you need for robotics: physics and affordance.

Our SOTA recipe specifically solves for:
• Affordance: model object details that convey functionality.
• Physics: calibrate simulation dynamics against real-world observations.

Our agent has the action space of a “full-stack simulation engineer” and jointly considers geometry, materials, colliders, articulation, and physics. It automatically calibrates physics against real trajectories and matches task-specific dynamics.

Simulation environments is useful only if it can provide meaningful policy level signals. Methods, comparisons, and results below.

[Chris Paxton @chris_j_paxton]

Yeah this is how robots are going to have to work in the future