Marion Lepert

PhD Student @StanfordAILab | BS/MS @Stanford, Olympian

[Marion Lepert @marionlepert]

Introducing Masquerade 🎭: We edit in-the-wild videos to look like robot demos, and find that co-training policies with this data achieves much stronger performance in new environments.

❗Note: No real robots in these videos❗It’s all 💪🏼 ➡️ 🦾

🧵1/6

[Marion Lepert @marionlepert]

@zuwang95 Thank you! In Fig 5, the Ours (no overlay) ablation does already correspond to Masquerade with no overlay + cotraining. The training pipeline was identical except that the video data used did not have overlays.

[Marion Lepert @marionlepert]

@dimadamen @jiaying_fang0 @leto__jean Thank you!

[Dima Damen @dimadamen]

This is one of the coolest ideas using EPIC-KITCHENS in a long while...
We've all been waiting to be replaced by robots! At least this is now done in the generative space...
Great work by @marionlepert @jiaying_fang0 @leto__jean @StanfordIPRL .. congrats!
https://t.co/O3139Zk6xg

[Marion Lepert @marionlepert]

Check out our full paper: "Masquerade: Learning From In-the-wild Human Videos Using Data-Editing"

Paper: https://t.co/NRh893zGaQ
Project page: https://t.co/rA5MYuh1rc

Grateful for my collaborators @jiaying_fang0 and @leto__jean!

🧵6/6

[Marion Lepert @marionlepert]

Introducing Masquerade 🎭: We edit in-the-wild videos to look like robot demos, and find that co-training policies with this data achieves much stronger performance in new environments.

❗Note: No real robots in these videos❗It’s all 💪🏼 ➡️ 🦾

🧵1/6

[Marion Lepert @marionlepert]

Masquerade outperforms baselines by 5-6x. Both robot overlays and co-training are indispensable for good performance.

🧵5/6

[Marion Lepert @marionlepert]

@JulienRineau_ We did not, but Rovi-Aug (closely related work for robot-to-robot transfer) did. They found they could avoid overlaying the virtual robot at inference by randomizing the lighting of the robot overlays during training.

[Marion Lepert @marionlepert]

Introducing Phantom 👻: a method to train robot policies without collecting any robot data — using only human video demonstrations.

Phantom turns human videos into "robot" demonstrations, making it significantly easier to scale up and diversify robotics data.

🧵1/9

[Marion Lepert @marionlepert]

@ChaoyiPan Thank you! Yes, although I think Phantom builds more closely off of Rovi-Aug, the authors' follow-up work on their original Mirage paper.

[Marion Lepert @marionlepert]

Really grateful for amazing collaborators Jiaying Fang and @leto__jean.

🧵9/9

[Marion Lepert @marionlepert]

Check out our full paper: "Phantom: Training Robots without Robots Using Only Human Videos"

Website: https://t.co/UbUcGr4AaD

Paper: https://t.co/ejIlP4XHyA

Our work builds on Rovi-Aug (https://t.co/7srhTWr1op), awesome work led by @Lawrence_Y_Chen and @Chenfeng_X.

🧵8/9