Michael Zeng

ML @sundayrobotics

Previously student at @MIT_CSAIL under Prof. @RussTedrake

[Michael Zeng @michzeng_]

@roblee_rl +1 to Sid and Abhinav

[Michael Zeng @michzeng_]

Action chunking — especially executing long action sequences open-loop — is widely used in imitation learning for robotic manipulation. Why is it so effective and do we really need it? We find a key reason:

Long open-loop execution helps short-context policies imitate non-Markovian experts.

With this insight, we show how to move beyond open-loop execution: extending policy context restores reactivity while achieving even higher task performance.

🧵(1/5)

[Michael Zeng @michzeng_]

Thanks for sharing! I'll definitely need to read more, but I like the idea of having this latent behavior state rather than setting obs/prediction/action horizons as hyper-params, although this seems challenging for arbitrarily long task horizons? I also wonder what happens if you run K-UBM fully reactively (i.e. re-plan at the highest possible frequency)?

[Michael Zeng @michzeng_]

Finally, a fun analogy for the paper: In the Chris Nolan movie Memento, Leonard can’t form new memories. To get anything done, he has to write his plans down: notes, photos, tattoos, with instructions for his future self to follow. This technique lets him keep moving forward in his search for his wife’s killer.

Modern imitation learning policies are very similar: they write down their plans in long action chunks and execute them blindly, open-loop. Why? Like Leonard, robot policies have little memory of the past, so they commit to a plan rather than continually try to figure out what to do next from limited information.

For Leonard, the notes are purely a compensation for an impaired memory. We think of long execution horizons much the same way: not something robot policies inherently need, but a workaround for limited context. Give the policy its memory back, and reactivity becomes possible again.

[Michael Zeng @michzeng_]

Is this a better paradigm for robotic imitation learning?

🌐 Website: https://t.co/xcgBHfsMVm
📄 arXiv: https://t.co/tlZqA1lv4h
💻 Code: https://t.co/aCcGOheJ2I

Huge thanks to my collaborators! @_abhinav_ag, @BatiAjay, @brianjsl, @siddancha, @RussTedrake

🧵(5/5)