Dwait Bhatt

Robotics x RL

[jack @jack]

We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.

https://t.co/r0qFLdvd6c

[Dwait Bhatt @dwait_bhatt]

@alexjswerdlow Very creative approach to critic learning, really cool paper!

[Alexander Swerdlow @alexjswerdlow]

In language models, turning a generative model into a critic is simple: pass in the output and prompt for a score. Robot policies can't do this - they output physical actions & there's no built-in vocabulary to predict scores.

So when it comes to RL on robot policies, we typically train a separate critic from sparse rewards - blind to everything the policy learned from the actions in the pretraining data.

We think this has it backwards - can a critic learn from those actions the way the policy does? (1/n)

[Octi Zhang @octi_zhang]

(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?

That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.

Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.

Paper: https://t.co/HZBYgG7SWK
Website: https://t.co/VyUKiYFI2y

[Jitendra MALIK @JitendraMalikCV]

I am seeing claims around LLMs, specifically Astra having made serious progress on robotics. The tasks that are demonstrated are simple pick and place tasks with parallel jaw grippers. LLMs can do planning, and the impressive demos in these tasks primarily show that. But robotics is also about dexterity and dynamics - which is why we need high frequency controllers/policies that can deal with torques and forces. So here is a simple challenge. Can you prompt an LLM to output the high frequency control commands for a legged robot in varying terrain e.g. https://t.co/JrbHBEWgVD RSS 2021, CoRL 2022 (this is by now 5 year old technology, so I am not picking a particularly hard task). I am not questioning the usefulness of LLMs for high level planning or in agentically assisting a robotics researcher (we use them all the time!).

[Huy Ha @haqhuy]

Transformer Transformer will be presented at #CoRL2026 🚀 See y'all in Austin!

Website: https://t.co/s7eimKcHbb
Video: https://t.co/YGQXUSZWLY
Paper: https://t.co/Biyu0r1J7N

[David Pantera @davidpantera_]

Two of us filmed this on a couple phones, and we used Atlas to turn it into a short film. This is just a clip from the @theworldlabs tech blog

Atlas is able to generate the parts of the scene that none of the cameras actually filmed.

It can freeze time, allowing you to reframe shots to positions never filmed, then resume.

View events from impossible angles because Atlas knows what was there.

[World Labs @theworldlabs]

Introducing Atlas:

The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D.

Model the world, move the camera, and simulate space & time.

[Dwait Bhatt @dwait_bhatt]

@seohong_park Thank you for the reference, it was fun diving into their analysis of this exact observation! Haha it's hard to keep up with all the relevant literature

[Seohong Park @seohong_park]

Behavioral cloning mystery

https://t.co/VqxzvcfGSx

I wrote a new blog post about "mysteries" in behavioral cloning that appear with real-world robot data (e.g., overfitting is "good"). I also tried to demystify them and shared my thoughts!

[Dwait Bhatt @dwait_bhatt]

@SkildAI This is amazing!
If this truly works for extremely diverse tasks, robots, and environments, then the last remaining piece is bringing up the 66% success on unseen tasks to 100% with some sort of continual learning / RL.
Exciting times!

[Sergey Levine @svlevine]

Latest Deep RL class lectures are now online!
https://t.co/GvqI1v3hgD
Thanks to @seohong_park, we now have CS185/285 for spring 2026 available to everyone to watch.

Course website here: https://t.co/U16rasTOAo

Apologies for a few recording glitches (it's not a perfect system).

[Andrew Wagenmaker @ajwagenmaker]

Action chunking is a critical component in virtually all modern approaches to imitation learning for robotics.

But why is it so critical, and do we really need action chunking? Check out our latest work to find out! (1/n)

https://t.co/s5AYfCoVpG

[Kun Lei @kunlei15]

RL-100 is now published in Science Robotics! The code is fully open-source at https://t.co/XxtXN1xx5m. Try training your first 💯 robot 🤖

[mimic @mimicrobotics]

From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1.

We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand.

The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.

[Dwait Bhatt @dwait_bhatt]

We are open-sourcing the entire project to enable the community to easily use hybrid strategies for online RL with demonstrations.

The repo provides a modular, extensible framework with clean implementations of all strategies in the paper (and a few more!).

In case you care about different metrics than SEI, you can also run your own analysis on our data made available through our public W&B project: ~2,500 training runs in total (pretraining runs + >100 hybrid online algos across 5 robot-task combos, >=3 seeds each).

🌐 Website (w/ interactive results - toggle tasks or strategies to see how the SEI distribution changes!): https://t.co/9giVD2oYdT
📄 Paper: https://t.co/pxi6DsXPLx
💻 Code: https://t.co/30AbaYklZa
📊 Weights & Biases: https://t.co/iVaIHMWmWL

🧵(7/7)

[Dwait Bhatt @dwait_bhatt]

The robot learning landscape today is heavily dominated by imitation learning (VLAs, diffusion). Because of this, RL is often viewed as just a way to improve a BC policy trained on demonstration data.

​In our ICRA paper, we shift to an RL-first lens and ask: how can we use demonstration data to maximize online RL sample efficiency? BC pretraining is just one piece of the puzzle. 🧵

[Dwait Bhatt @dwait_bhatt]

Further analysis:
1️⃣ We isolate the impact of individual components (e.g. buffer prefilling, source of demonstrations, offline RL, residual RL) on sample efficiency.
2️⃣ We evaluate how the SEI distribution shifts as the tasks get harder.

Takeaways:
• RLPD-style buffer prefilling has the most consistent positive impact on sample-efficiency.
• Conservative critic pretraining mostly hurts sample efficiency, due to offline-to-online recalibration.
• Critic pretraining becomes important as tasks get harder - e.g. for the most difficult task considered (StackCube), the most efficient method uses critic pretraining with our simple MCQ baseline.

🧵(6/7)