Rickmer Krohn

Multisensory Reinforcement Learning for Robots | PhD student at TU Darmstadt |

[Nico Bohlinger @NicoBohlinger]

⚡️What if we could design robots with gradients?

🤖 Introducing Shape Your Body: we train one multi-embodiment policy + value function, then optimize new robot designs through value gradients.

🔗 Try out our interactive demo here:
https://t.co/hSpc7qcyqP

[Georgia Chalvatzaki @GeorgiaChal]

🤖 @ieee_ras_icra is almost here, and the PEARL Lab is bringing the whole crew to Vienna! Here is where to catch us. 👇

🗓️ Monday is workshop day, and a busy one for me: three invited talks and two panels.
• 09:50, Stolz 2: invited talk at “Dexterity with Multifingered Hands,” then the panel at 11:40
• 11:00, Hall A1: invited talk at the RL in the Era of Imitation Learning workshop
• 14:30, VIP Lounge C: “Structured Robot Learning for Rigorous Manipulation: From Perception to Action” at the Rigorous Robot Perception workshop, then the roundtable at 16:30

🥊 And to close the week, on Friday, I am on the “Geometry in the Age of Data-Driven Robotics” debate, Friday 11:00 in Hall C4 (full panel at 16:00), arguing the contra side of the claim: “The vast majority of interesting research questions on the role of geometry in robotics have already been answered.” Come heckle.

But the part I am most excited about is the team. Their work is everywhere this week.

📄 Main conference (Interactive Sessions, Hall C, 09:00 to 10:30):
• Tue: @AoiAoi83259839 presents “Stein Variational Ergodic Surface Coverage with SE(3) Constraints” (TuI1I.226)
• Wed: @an_thai_le presents “Global Tensor Motion Planning” (WeI1I.68); @FrankieZhang41 presents “Adaptive Diffusion Constrained Sampling for Bimanual Robot Manipulation” (WeI1I.112); @yjin_1118 and @FatAndFurious42 present “SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation” (WeI1I.194)
• Thu: @SnehalJauhri @SophieLueth present UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scene (ThI1I.226)

🧑‍🔬 In the workshops:
Monday
• @SnehalJauhri, “UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scene” @ MM-Spatial AI for Robust Navigation and Open-World Understanding (Lehar 3) AND Act to Sense to Act Better (Hall C4)
• Magnus Dierking, “Real-World Deployment of Massively Parallel Sampling-Based MPC for Contact-Rich Manipulation” @ Frontiers of Optimization for Robotics (Lehar 4)
• @kumarmanas27, “Vision-Language Modeling for Natural-Language Wheel Loader Assistance in Unstructured Construction Environments” @ Long-term Deployments in the Wild (Strauss 2)
• @kumarmanas27 & Franziska Herbert, “Hierarchical Scene Graphs and Contact-Aware Behavior Trees for Bimanual Manipulation” @ CR2 (Lehar 2)
• @RKrohn_, “Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning” @ CR2 (Lehar 2)
• @AoiAoi83259839, “IMPACT: An Implicit Active-Set Augmented Lagrangian for Fast Contact-Implicit Trajectory Optimization” @ CR2 (Lehar 2)
• Franziska Herbert, “Semantic-Geometric Task Representations for Bimanual Manipulation from Human Demonstrations to Robot Action Planning” (RA-L) @ Shared Challenges in Human-Centered and Resilient Robotic Autonomy (Hall C6)

Friday
• @SnehalJauhri , “MolmoBot: Large-Scale Simulation Enables Zero-Shot Manipulation” (spotlight!) and “Whole-Body Mobile Manipulation using Offline RL on Sub-optimal Controllers” (WHOLE-MoMa) @ Beyond Teleoperation (Strauss 3)
• @SophieLueth, “CoFlow: Residual Hierarchical Flow Matching for Effective Physical Human-Robot Collaboration” @ Bridging the Gap between Robot Learning and HRI (Hall A2)
• Max Siebenborn, “Morphologically Equivariant Flow Matching for Bimanual Mobile Manipulation,” spotlight @ Geometry in the Age of Data-Driven Robotics (Hall C4)
• @RKrohn_, “Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning” @ Manipulation Robustness (Hall A3)
•@FatAndFurious42, “Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning” and Franziska’s RA-L paper @ Semantics for Reliable Robot Autonomy (VIP Lounge C)

If you are in Vienna, please stop by our talks and posters. We would love to chat. 👋 #ICRA2026

[Rickmer Krohn @RKrohn_]

What a first day at IEEE ICRA! Happy to show my recently accepted work about multisensory reinforcement learning and that @GeorgiaChal talked about my recently submitted paper Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation!

[Rickmer Krohn @RKrohn_]

Accepted at IEEE Robotics and Automation Letters 🎉
With @FatAndFurious42 @GabrieleTiboni1 @GeorgiaChal
🌐 Website + videos: https://t.co/EXrgomIdsc

[Rickmer Krohn @RKrohn_]

Solving contact-rich manipulation tasks even under disco light🪩?
Incorporate multiple sensors using MSDP and obtain a robust policy in under 55 min!
Excited to share our work accepted at RA-L: Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning

[Rickmer Krohn @RKrohn_]

⚡ Full pipeline (data + pretraining + online RL) in <55 min
🤏 Only 6,000 real-robot interactions needed
📈 FT-sensor boosts success rate by +14%
🦾 ~90% success on challenging contact-rich tasks
Robust to: sensor noise · variable stiffness · external forces · 🪩 disco lights

[Rickmer Krohn @RKrohn_]

Multisensory Dynamic Pretraining (MSDP) is based on masked autoencoding and cross-sensor prediction, leading to rich sensor fusion. Usage of the FT-sensor boosts performance by 14%! Our approach is flexible regarding the number and type of sensors. More Details in the Preprint!

[Rickmer Krohn @RKrohn_]

Real-world online Reinforcement Learning from Vision + ForceTorque + Proprioception --> With MSDP we achieve ~90% success rate in only 35min of online training!

Checkout the Preprint here: https://t.co/bmme0ESX0v

Thanks to @FatAndFurious42 @GabrieleTiboni1 @GeorgiaChal!