zeji yi

phd @CarnegieMellon | Intern at @Amazon FAR| Previously @Tsinghua_Uni | Studying Control & Robot Learning

[zeji yi @ZejiYi]

Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞.

❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings.

Here's why 🧵

[Yisong Yue @yisongyue]

@YananSui’s group built this.

[C. Zhang @ChongZzZhang]

With @JayHe748646 @KehanWen170077 @leggedrobotics , we push humanoid mobility to next frontier.

Achieved by a generalist contact planner + a generalist tracker + a generalized neural mapping, warmstarted from few human data.

Website: https://t.co/Y7cu3D4K4I
Promo Video 👇

[zeji yi @ZejiYi]

@roblee_rl Yeah, we’ve tried different chunk lengths, but even with discrete actions, reasonably long chunks still seem necessary. Still a bit of a mystery!

[zeji yi @ZejiYi]

Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞.

❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings.

Here's why 🧵

[zeji yi @ZejiYi]

Thanks! That’s a sharp observation. By “sensible actions,” do you mean lower EEF MSE on the validation set? If so, I agree, but lower MSE doesn’t necessarily translate into higher rollout success rates.

I also wonder how much this depends on training scale. The >99% training-set token accuracy requirement might be specific to smaller-scale settings; at VLA scale, even training-set accuracy may never reach that level. When comparing rollout success rates rather than MSE, I’m not convinced discrete policies necessarily learn more slowly than continuous ones even in earlier stages.

[zeji yi @ZejiYi]

We observed something similar: CFG provided only modest gains for continuous flow in our experiments. That might partly explain why these WAMs omit action-head CFG. Standard CFG also requires an extra unconditional evaluation at each step, which is a burden for inference as well.

Our interpretation is that CFG helps DR much more than DF because it sharpens token distributions, reducing mode ambiguity and the resulting factorization error. DF already mitigates this through iterative refinement, leaving less room for CFG to help. So even a single application of guidance can make a big difference for DR.

[zeji yi @ZejiYi]

Cool idea! TBH, we haven’t tried that yet—we kept the CFG implementation consistent across heads for a controlled comparison. Agreed that the unconditional DR predictor should recover the empirical bin frequencies, though jointly learning that prior might still benefit the shared network.

[zeji yi @ZejiYi]

@max_simchowitz Thank you, Max! Truly so lucky to work with you! 💛

[zeji yi @ZejiYi]

Good question! The key insight is that, except for tasks requiring extremely high precision, such as Tool Hang in RoboMimic, 256–512 bins per action dimension are generally sufficient, and even 128 bins work well for many tasks. These often correspond to a spatial resolution of roughly 0.2–0.4 cm for 256–512 bins, or 0.8 cm for 128 bins. However, reducing the number of bins further can lead to noticeable precision loss.

[zeji yi @ZejiYi]

@m_wulfmeier Hah, spoiler of future work 🤣

[zeji yi @ZejiYi]

@HaoruXue Hopefully it’s not just a trailer🤣, but I’m looking forward to your new work!

[zeji yi @ZejiYi]

@peterxichen thanks peter 🥳

[zeji yi @ZejiYi]

12/12 Paper: https://t.co/lGR150SiFw
Project page, 5-min explainer and all rollout videos: https://t.co/H339nx4ESR
Code and checkpoints: coming soon.

Joint work with @AstonishingWolf, Xinyang Zhang, @KoushilSreenath, @guannanqu, @peterxichen, @max_simchowitz, @GuanyaShi at CMU and Amazon FAR.

[zeji yi @ZejiYi]

11/12 Takeaway: we may need to 𝐫𝐞𝐭𝐡𝐢𝐧𝐤 𝐰𝐡𝐞𝐭𝐡𝐞𝐫 𝐫𝐨𝐛𝐨𝐭 𝐚𝐜𝐭𝐢𝐨𝐧𝐬 𝐦𝐮𝐬𝐭 𝐛𝐞 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬.

Uniform bins, the simplest discrete actions, match or beat continuous heads once their tokens are kept consistent, through guidance or iterative refinement.

Perhaps robot actions are 𝐧𝐨𝐭 𝐭𝐡𝐚𝐭 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭 𝐟𝐫𝐨𝐦 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐚𝐧𝐝 𝐯𝐢𝐬𝐢𝐨𝐧: they can be tokens too.