More sim2real RL, this time a longer horizon task.
I especially like how this policy pushes the two base cups together while holding the top cup. Sparse reward RL is all that is needed to discover these strategies!
@sydekickbot
More sim2real RL, this time a longer horizon task.
I especially like how this policy pushes the two base cups together while holding the top cup. Sparse reward RL is all that is needed to discover these strategies!
@sydekickbot
@TimXu222575@MarioChan2002@HaoruXue The issue I think is that many of the models that are benchmarked are trained on (mostly) real world data, which then likely doesn't translate well to this sim benchmark, and so such models will be under represented
Working with RL is always full of fun surprises. This policy learned to adjust its grasp using contact with the table. RGB policy, trained entirely in sim with sparse reward.
More soon @sydekickbot
@iamorphen Yeah, I can't quite tell what it was trying to do when it closed the gripper, since it ended up deciding to use the gripper to rotate the bottle and grasp it.
pendulum has fully swung back from where it was last year (“scale up data collection, behavior clone large models”). everyone is finally seeing the shining light of RL!
@micoolcho@sippeyxp Yeah, I think latency can be manageable in many situations, but there's also many robotics applications that might not have access to great networks.
@joshgruenstein This is super cool, great work, and thanks for the detailed article. I'm wondering why you chose diffusion for the visuomotor policy. I assume the privileged-state RL actor was uni-modal, but did diffusion still provide some other benefits?
"the bitter lesson comes for robotics" and it's literally a hand-designed architecture with its every component and hyperparameter painstakingly ablated against human-specified performance metrics that is then initialized by standing on the shoulders of humans as it consumes digitized civilizational knowledge of immense scale before it gets slow-cooked during an extended phase of further distilling human knowledge from pristine resources only to then be relentlessly beaten against the well-tuned reward functions of a million partly crowd-sourced and partly prompted-into-existence environments that humans decided were worth hillclimbing in the current quarterly iteration of the human-orchestrated cycle of intelligence creation