(๐ต๐๐๐๐๐๐๐ข๐๐ ๐๐ข๐ ๐๐: ๐ ๐๐๐ก๐ก๐๐ ๐๐ข๐๐ก ๐๐๏ฟฝ๏ฟฝ๐ฆ๐๐ ๐๐ฆ ๐๐ ๐๐๐ ๐กโ๐ ๐๐๐๐๐ก ๐๐ก๐ ๐๐๐ ๐ถ ๐ธ๐๐๐๐ฆ ๐กโ๐ ๐ฃ๐๐๐๐ ๐ค๐๐กโ ๐ ๐๐ข๐๐ ๐๐! ๐)
Ever dreamed of a robot that could play the harp ๐ผ, tie your shoelaces ๐, or even perform delicate wet-lab experiments ๐งช?
The key is ๐ฅ๐๐๐ซ๐ง๐ข๐ง๐ ๐ง๐จ๐ญ ๐ฃ๐ฎ๐ฌ๐ญ ๐ก๐จ๐ฐ ๐ก๐ฎ๐ฆ๐๐ง๐ฌ ๐ฆ๐จ๐ฏ๐, ๐๐ฎ๐ญ ๐ก๐จ๐ฐ ๐ญ๐ก๐๐ฒ ๐๐๐๐ฅ ๐๐ง๐ ๐๐จ๐ง๐ญ๐ซ๐จ๐ฅ ๐๐จ๐ซ๐๐๐ฌ. We introduce ๐ ๐๐๐, a ForceโActionโTrack foundation model that learns from 2,000 hours of force-rich human demonstrations in our EgoForce dataset, bringing us closer to robots with truly human-like dexterity! ๐ฉท๐๐
https://t.co/iR7NQXBELs
Bigggggg shoutout to @ZhiCao01756604 !
๐ Excited to share T2-z: Test-Time Optimization of Environment Latents for Robotic World Models!
Same observation, same action, different environments โ different outcomes.
How can world models adapt to hidden environmental factors like friction, mass, or physical rules?
๐ก T2-z conditions world models on optimizable environment latents.
T2-z directly optimizes the conditioning latents through world prediction, without supervision of what these factors are, what values they take, or how many factors exist.
๐ Meaningful latent structure emerges purely through world prediction! For compounded environmental factors (mass ร friction), PCA of the learned latents reveals a two-dimensional structure that closely reflects the underlying factor distribution, despite the model never knowing how many factors are present or what they represent.
๐ค Across 7 simulated settings and 4 real-world robotic tasks, T2-z:
โขReduces object prediction error from 15.20 to 7.00 (54% reduction).
โขImproves action success from 33.73% to 72.96% (+39.23 percentage points).
๐ Project: https://t.co/AvZ8IauxLC
๐ Paper: https://t.co/ZifSeQfeby
๐ป Code: https://t.co/nfJRaQRVfp
Collab with Chenyuan, @RavenHuang4@YunfanJiang@wenlong_huang@davheld@vitorguizilini@yuewang314@drfeifei@jiajunwu_cs , leo and @YejinChoinka