Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state.
Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides:
- Known Geometry: 16 rigid functional surfaces
- Surface Motion: 22 anatomically aligned joints
- Tactile Contact: 2048 piezoresistive taxels
Huge thanks to my advisor Ding @zhao__ding, and to Yuxiang @yxyang1995, Maria @bauzavillalonga, Marissa, and Peide @peide_huang for the valuable advice and discussions.
Paper: https://t.co/xSYCyqHrZf
can someone plz make a head-mounted data capture device designed to be forgotten about while building hardware
hardware req:
- lightweight
- all day life, can have a usb-c port for external charging dont even need battery onboard, can even ship with a toolbelt with the battery even
- i dont wear glasses or hats, so ideally doesn't need to be either of those form factors, though id prefer glasses to hats
- audio, high quality cameras (include thermal camera for debugging pcbs/motors/robots), high quality enough for detailed playback/posting
- speaker (bone conducting ones)
- always on recording (dont want to ever miss recording something bc i forgot to press record)
- physical clip button
- looks like something tony stark would wear
software
- streams to laptop/phone only, rolling buffer of whatever time i choose
- view clips from my laptop/phone instantly whenever i want to
- rolling qr code of timestamp so i can sync the camera stream to some hardware experiment im doing just by looking at the screen
- can seamlessly integrate into my dev workflow (like wispr) so i can talk to claude code handsfree and ideally have it share context with what im seeing and hearing (make the device firmware itself dead simple and have the algo that determines if im done talking run on the laptop/phone).
i'll pay a lot for even a prototype that works.
The first time we rolled a robot into a new warehouse, it didn’t perform as well as we expected.
It took us an entire day of debugging, before we realized it was something simple… the cameras were wired completely wrong. 🤦
Left camera to right gripper 🔀 right camera to left gripper.
But what was interesting was, the model still kind of worked.
In post-training data, left side = bin, right side = conveyor. But even when swapped, the model would still do the task—just slightly worse. Enough to fool us into thinking it was something else for hours… until the moment we switched the cameras back. Then it worked great.
This wasn’t the first time we’ve seen emergent ambidexterity.
During a packing demo last year, we spotted the robot using the “wrong” hand to shake a USB brick out of a tight baggie. Totally outside the post-training data (we watched all 17 hrs of it to double check). 100% left hand, but for some reason at inference time, it felt the need to use the right hand. Nothing in the model architecture could obviously explain this kind of invariance.
If these models are headed where I think they are, imagine one day having a generalist “substrate of intelligence” where you can plug in any number of sensors and actuators, and the whole thing just springs to life.
It wouldn’t matter how you wired it up. It would just work.
That would be pretty cool.
@sentientcar@GeneralistAI Really cool! What’s the top wrist speed your slam can do before losing tracking? Would assume adding imu would help with this. I notice you aren’t rotating the umi in the video, can your slam track well with heavy rotations?
Curious why the policy decided to open the bag at 0:52. The bag seems to look empty from the viewpoints of both wristcams. Policies need efficient memory architectures to solve occlusions well. Really amazing demo!
But it’s bad to rely on swapping alone. There is likely materials we haven’t discovered yet or ones under the radar that wear at slower rates where it’s much less of a problem. We must find them and commoditize them with cheap robots.
Companies like https://t.co/VVNrXaFPpW have developed skin that can last millions of cycles under constant wear.
But it’s bad to rely on swapping alone. There is likely materials we haven’t discovered yet or ones under the radar that wear at slower rates where it’s much less of a problem. We must find them and commoditize them with cheap robots.
Companies like https://t.co/VVNrXaFPpW have developed skin that can last millions of cycles under constant wear.
This is GEN-1 putting paper bills into a wallet.
Paper has always been deceptively hard for robots.
Thin, deformable, and unforgiving—it bends, folds, and slips.
Not precise, you miss. Too much force, you crumple.
Easy for humans. But for robots, it’s a full-stack challenge.
@JieWang_ZJUI@sincethestudy High quality controls, IK, middleware makes sure your hardware is at the right place at the right time and affects everything from teleoperator cognitive load to hardware lifespan.
Data, models, and evals are always downstream.