1/ Progress in robot locomotion scaled with simulation, but touch was left behind. Our work, Tactile Genesis, unlocks sim for dexterous manipulation. Using sim to ablate tactile types and models, we can quickly converge to better tactile hardware and force-informed policies.
It is a hard and sad decision. I shared this message with folks at Thinky. Thank you all for the time together♥️ Just as the last sentence in my message: The future worth building is human.
1/ Progress in robot locomotion scaled with simulation, but touch was left behind. Our work, Tactile Genesis, unlocks sim for dexterous manipulation. Using sim to ablate tactile types and models, we can quickly converge to better tactile hardware and force-informed policies.
Presenting PTLD: an approach to learning tactile dexterous policies without ever simulating the tactile sensor.
Tactile is essential for performing highly dexterous manipulation. However, collecting tactile observations reliably has been a fundamental bottleneck: (1) teleoperating a multi fingered hand for dynamic tasks is challenging, making sim-to-real imperative, (2) one can’t realistically simulate tactile today
Robotics people, follow @aran_nayebi . He's been asking some really fundamental questions about what morphologies we need for robotics, and comes from a really unique bio/neuro perspective. Check this work out!
@cspaliwa1 I have limited experience with real tactile hardware, I only know what I've read and heard. I'll leave the exploration up to the community. Upcoming conferences will have tons of tactile papers.
@Em_Nomadic Yes! This is the true value of good simulation :) not just as a worse/cheap fallback for reality, which is the way policy training people see sim as..
@_varunnair@nvidia It's funny because IsaacSim actually does have a tactile sensor, but it was integrated quietly. Mimics GelSight, works on flat pad only https://t.co/F9y2yzuyrq
9/ Thanks to my advisors @aran_nayebi@KaterinaFragiad and collaborators @kashu_yamazaki@gs_ai_ for supporting me throughout this work! ❤️ Follow them for more cutting edge NeuroAI and robotics research🧠🦾
🚀 New Open-Source Release! PyTorchTNN 🚀
A PyTorch package for building biologically-plausible temporal neural networks (TNNs)—unrolling neural network computation layer-by-layer through time, inspired by cortical processing. PyTorchTNN naturally integrates into the Encoder-Attender-Decoder (EAD) architecture (Chung*, Shen* et al., 2025), which flexibly combines diverse neural networks, motivated by the fact that no single model (Transformer, SSM, RNN) dominates all sequence learning tasks.
🧵👇
🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate?
🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch.
🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks.
Code is fully released; check out our:
Homepage: https://t.co/I8kVJO3XPw
Paper link: https://t.co/BFqpOwXYbT
Github: https://t.co/345eTpTx0y