Thrilled to announce our paper Dynamem has been selected as the 𝐁𝐞𝐬𝐭 𝐏𝐚𝐩𝐞𝐫 at the Lifelong Learning for Home Robots Workshop at #CoRL2024! Thank you for your support.
I am currently an intern at @hellorobotinc and will keep optimizing Dynamem there.
If you want robots that can just live with you & help 24/7, it needs to build & update its memory on the fly. Current semantic memory representations like VoxelMap from OK-Robot can't change with the world.
That's why we built DynaMem: dynamic memory for a changing, open world!
Excited to share Do as I Do! We turn everyday human videos into physically consistent robot data that can be directly executed in the real world.
This was a fun collaboration with @bhawna_paliwal_ and @willjhliang, with lots of moving parts. More details in Mahi's thread below👇
Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data?
Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵
Turns out you can train humanoid hands without any robot data.
The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors.
We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies.
FACTR 2 consists of:
1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors.
2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training.
w/ @StevenOh_@_tonytao_
🧵(1/N)
Force sensing for low-cost robot arms — without adding force sensors.
🚀 Excited to share FACTR 2! 🚀
FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning.
w/ @JasonJZLiu@_tonytao_
🧵(1/6)
"There’s a whole set of new features, some hidden, some very obvious, that just make this fundamentally a total redesign, top to bottom."
Thanks to The Robot Report for a great writeup on the all-new Stretch 4!
Read more: https://t.co/TFsmwqZU9R #HelloRobot#robotics
I am very proud to work here when the team was working on this robot. Excellent onboard perception, providing both wide field of view along with accurate depth prediction. All of the work that works on Stretch 3 is going to be significantly boosted on this version.
Say hello to Stretch 4! 📷📷
The team at #HelloRobot isso excited to announce the release of our new robot! It's been an absolute labor of love, and we couldn't be prouder to introduce the next evolution of Stretch.
Explore our brand-new site: 📷https://t.co/EgTaYz6kGZ
Actually, even Siglipv1 is strong enough to match object features, not just with another object feature, but also with the text feature. This idea will give you a language-conditional robot navigation system https://t.co/lSJhDdgcEv
We noticed that DINOv3 was surprisingly strong at matching object features.
This inspired L2G (Local Matches to Global Masks). With a few reference images, a robot can search a room for the target object.
🔗 Project: https://t.co/PTy0C8xlTS
💻 Code: https://t.co/QWRP5IBEd3
Why buy a robot when you can build your own?
Meet YOR, our new open-source bimanual mobile manipulator robot – built for researchers and hackers alike for only ~$10k. 🧵👇
It’s hard to find true zero-shot end-to-end policies – ones that work without any fine-tuning in fully novel, simulated environments, even for single tasks! We test two policy families, the π family from @physical_int and the recent Contact-Anchored Policies (CAP) from NYU & UCB.
On all our tasks, we are making steady progress – but we are nowhere close to saturation yet.
DynaMem (https://t.co/lSJhDdgcEv) ran in the background of this video. It combines picking, placing, and navigation skills together to solve long horizon mobile manipulation. Integrating CAP can definitely improve the manipulation robustness and speed of DynaMem significantly!
CAP 🧢 works well on our academic data, compute, and parameter budget – training 3 general policies for pick, open, and close on only 23 hrs of data. Fun fact: one of them has already won a best demo award in CVPR'25 after doing picks all day. It's only gotten better since then (4/n)
We just released AINA, a framework for learning robot policies from Aria 2 demos, and are now open-sourcing the code: https://t.co/HSHrtUrt11. It includes:
✅ Aria 2 data processing into 3D observations like shown
✅Training of point-based policies
✅Calibration
Give it a try!
Dexterous manipulation by directly observing humans - a dream in AI for decades - is hard due to visual and embodiment gaps.
With simple yet powerful hardware - Aria 2 glasses 👓 - and our new work AINA 🪞, we are now one significant step closer to achieving this dream.
When @anyazorin and @irmakkguzey open-sourced the RUKA Hand (a low-cost robotic hand) earlier this year, people kept asking us how to get one.
Open hardware isn’t as easy to share as code.
So we’re releasing an off-the-shelf RUKA, in collaboration with @WowRobo and @zhazhali01.
Glad to intern at Hello Robot Inc. If you are interested in trying this Embodied Question Answering system at your home, feel free to check the codes in Stretch AI https://t.co/20BDNpxOpi
Our former intern, Peiqi Liu's project enables Embodied Question Answering (EQA) by combining Gemini 2.5 Pro with the dynamic semantic memory framework from DynaMem, allowing Stretch 3 to interpret and answer questions about its environment in real time. 🤖https://t.co/agdkOx601n
Just arrived at Hangzhou for #IROS2025
I’ll present Neural MP at TuAT1.1 (Award Finalists Session 1), super excited that we got nominated as Best Paper Finalists and Best Student Paper Finalists!
🗓️ Oct. 21, 10:30-10:35 AM
📍 Room 401
🔗 Neural MP https://t.co/hwjB3mg9Ud
I’ll also give a spotlight talk for DRP at the LeaPRiDE workshop!
🗓️ Oct. 20, 10:10-10:20 AM
📍 Room 102A
🔗 DRP https://t.co/FwV3LomfK9
🔗 LeaPRiDE Workshop https://t.co/BeJfnRiyOG
Looking forward to meet old and new friends!
I gave a Early Career talk at CoRL 2025 in Seoul last week, where I talked about my observations from the past decade in robot learning along with where the field is headed for the next decade.
In summary, the future of robot learning needs:
(1) Data beyond teleop: We are never going to reach the scale of LLM / VLM data by tele-operating robots. Need to leverage consumer hardware already in people's hands (e.g. iPhones) and emerging devices (e.g. Smartglasses).
(2) Observations beyond vision: The hard problem in robotics is dexterity. Dexterity is all about moving objects intricately through contact. The sense of touch is critical for this. Vision can help you acquire objects, but anything more complex will need touch.
(3) Reasoning beyond reactivity: The biggest wins in robot learning have been in reactive policies (both manipulation and locomotion). But the class of models that got us here are generally feed-forward nets. Long-horizon reasoning needs the ability to predict future outcomes and manipulate them. Currently unclear what the right scalable architectures are here, but we are working on it.
(thanks @zacinaction for the pic!)
🎉We will be presenting GraphEQA at #CoRL2025 in Seoul!
Curious about how to utilize 3D scene graphs for context-aware navigation in unexplored 3D environments? 👋Come visit us @ Spotlight 2 on Sept 28 (Poster 66)!
Website: https://t.co/xmYqEWC4Gw
Code: https://t.co/2qQP6qME08
We built a robot brain that nothing can stop.
Shattered limbs? Jammed motors? If the bot can move, the Brain will move it— even if it’s an entirely new robot body.
Meet the omni-bodied Skild Brain:
From dexterous hands to imitation from internet videos, his group keeps dropping breakthroughs that set the tone for the field.
@LerrelPinto’s lab at NYU has quietly
reshaped robotic learning.
A breakdown 🧵
[📍SAVE MEGA THREAD FOR LATER📍]