When a robot learns to react to touch, where does the training data come from?
Researchers from @UCBerkeley, @nvidia, and @Stanford introduce T-Rex, a framework that unifies vision, language, and tactile sensing so robots can respond to physical contact in real time rather than relying on vision alone.
On contact-rich tasks like inserting a card, turning a key, and handling deformable objects, it outperforms the strongest baseline by more than 30% across 12 real-world tasks.
The foundation is a 100-hour tactile-synchronized teleoperation dataset spanning 200+ everyday objects and 22 motor primitives. During data collection, researchers wore @ManusMeta gloves to capture precise finger motion, which was then retargeted onto @SharpaRobotics Wave dexterous hands for bimanual teleoperation.
Learn more: https://t.co/Kz85DsTxR0
Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state.
Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides:
- Known Geometry: 16 rigid functional surfaces
- Surface Motion: 22 anatomically aligned joints
- Tactile Contact: 2048 piezoresistive taxels
Huge thanks to my advisor Ding @zhao__ding, and to Yuxiang @yxyang1995, Maria @bauzavillalonga, Marissa, and Peide @peide_huang for the valuable advice and discussions.
Paper: https://t.co/xSYCyqHrZf