Rob Lee

deploying useful robots @sydekickbot. interested in RL, IL, humanoids etc.

[Rob Lee @roblee_rl]

IMLE Policy introduces a new way to train faster and more data efficient behavior cloning policies.
Will be presented at RSS2025!

https://t.co/y8ZTPxwdLf

🧵⬇️

[Rob Lee @roblee_rl]

perpetual cup stacking @sydekickbot

[Rob Lee @roblee_rl]

@joudanki 😂me making coffee every morning

[Rob Lee @roblee_rl]

@kevinmpeterson1 @JCChristopher Nice. Do you see a future where you get to deployment with the modular approach, and then train end-to-end with the data from the real deployment data later on? Does that data scale well in construction settings?

[Rob Lee @roblee_rl]

less than 1 hour of data. i love policy eval timelapses :)

[Rob Lee @roblee_rl]

@Saketh_Vaishya it's the flexiv rizon https://t.co/WlKzeOSCH9

[Rob Lee @roblee_rl]

a good model trained on even a simple task with a tiny amount of data feels mesmerising, no matter how many times you see it.

[Rob Lee @roblee_rl]

@Saketh_Vaishya it's an end-to-end learned policy!

[Chris Paxton @chris_j_paxton]

Seeing a model work like this for the first time is such a good feeling

[Rob Lee @roblee_rl]

@sentientcar We use both! This arm is 7dof, has a larger workspace, and a higher payload, which is nice for certain applications

[Rob Lee @roblee_rl]

@carlosdponx @randallmbriggs @GoingBallistic5 Westwood Robotics BEAR actuators also have liquid cooling, used in their humanoids

[Rob Lee @roblee_rl]

@ed0henderson Or more simply, the loss might look like it has plateaued, but the model might still be tweaking the smaller precise parts of the movements.

[Rob Lee @roblee_rl]

@ed0henderson Imitation learning is weird because there's no great way to pick checkpoints other than eval perf. It's hard to pinpoint overfitting since human demos are noisy/multimodal/not iid. Especially with small datasets the val set might not be perfectly representative either.

[Rob Lee @roblee_rl]

@ZakharovSergeyN Awesome work!

[Rob Lee @roblee_rl]

@Goodeat258 Nice, thanks! Do you plan to release the code for the robotics experiments? Curious how you create the positive/negative samples for imitation learning (since theres only one label per conditioning)

[Rob Lee @roblee_rl]

@asimovinc This might be useful, you only need a few simple terms for nice gaits (single foot contact, airtime, simple penalties) https://t.co/buQRfjoMhx

[Rob Lee @roblee_rl]

@JieWang_ZJUI Interesting. I guess their contribution is more around training data/recipe?

[Rob Lee @roblee_rl]

Right, that section shows averaging the outputs of a flow policy doesn't hinder performance much. They also show a figure with very minor spread of modes. In my experiments I found similar behavior, but there are definitely states where diffusion will output multiple modes. In most cases though, you can still get good success rate while collapsing modes, because you will often move to a state with less action ambiguity. (https://t.co/tjyxR01q7f) It's highly dependent on task and dataset though

[Rob Lee @roblee_rl]

Great work! Really interesting paper. I'm curious about what you think about recent non-iterative generative policies (C1+C2) like https://t.co/z26zf4YI4f and https://t.co/nrjjOAf9cN. These methods are basically regression but with additional mechanisms that encourage better use of the noise space. It seems that either C1+C2 and C2+C3 can work well, but I wonder about the trade offs.