E Harrison

EECS @ UC Berkeley | Robotics + ML @ BAIR

[E Harrison @ehharrison4]

Thanks also to all the collaborators: Philipp Schmitt, Bernd Kast, Philine Meister, @pranav_atreya, @qiyang_li, Finn Ferchau, Cesar Colmenero, Yash Shahapurkar, Gokul Narayanan, Melih Erdogan, Kai Wurm, @GeorgWichert, @oier_mees, Eugen Solowjow, @ajwagenmaker, @svlevine
(n/n

[E Harrison @ehharrison4]

It’s been great leading this work with Brian Zhu, Momen Khalil and Emanuele Poggi from Siemens Research and Predevelopment in the US and Germany, along with those at UC Berkeley, ETHZ, and Microsoft to make this project happen.

Full paper: https://t.co/9kfNxyLjuF
(6/n)

[E Harrison @ehharrison4]

The success of ARLI on real-world tasks over synchronous RL highlights an important principle for effective fine-tuning: not only do we need ARLI to enable fine-tuning with asynchronous inference, we actually need asynchronous inference to enable fine-tuning too.
(5/n)

[E Harrison @ehharrison4]

We evaluate ARLI across simulated and real-world manipulation tasks and find that it enables effective RL fine-tuning in settings where standard RL can fail entirely—while even matching or exceeding standard RL in idealized no-latency settings.
(4/n)

[E Harrison @ehharrison4]

We propose ARLI, a latency-aware framework that allows generalist robot policies to be RL fine-tuned despite inference delay.

By adding committed actions and intermediate observation to the RL state, the resulting policy achieves reactive capabilities despite VLA latency.
(3/n)

[E Harrison @ehharrison4]

Asynchronous VLA inference reduces inference delay, but breaks the Markovian assumption necessary for RL fine-tuning.

How can we enable RL fine-tuning of VLAs with async inference? We introduce ARLI: Asynchronous RL with Intermediate Information!

https://t.co/5A2OrJcaUH
(1/n)

[E Harrison @ehharrison4]

Modern VLAs are becoming increasingly capable, but large inference latency creates a major challenge for RL.

When a robot must wait to generate its next action, the resulting delay can alter the effective dynamics of the environment and undermine standard RL assumptions.
(2/n)