Billy Yan

@CILVRatNYU #RobotLearning #ComputerVision

[Mahi Shafiullah 🏠🤖 RSS 2026 ✈️ @notmahi]

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data?
Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

[AgileX Robotics @AgilexRobotics]

Congrats to @nyuniversity @Tsinghua_Uni and @UMich team on #HUG (Human Universal Grasping), and proud that our #NERO 7-DoF #robotics arm served as the core mobile manipulation platform!

HUG achieves zero-shot dexterous grasping 👉https://t.co/Q9w4zTK1bW

[WUJI TECH @wuji_global]

Human data → robot skills. Excited to see more work built on large-scale human demonstrations to advance dexterous manipulation. Great work from the team!

[Dandan Shan @DandanShan_]

🧐A question I've long been interested in: how can we learn from human hands and transfer that directly to robots?

Our new work, HUG, makes it possible in three simple steps: (1) collect human grasps at scale, (2) learn from them, and (3) retarget for deployment.

[Irmak Guzey @irmakkguzey]

Human data offers far greater generalization than robot data can.
In our new work, HUG, we show how powerful this claim can be.
Here's HUG: trained purely on egocentric human data (1M object-grasp pairs), deployed directly on the robot; zero-shot, in-the-wild.

[Lerrel Pinto @LerrelPinto]

Turns out you can train humanoid hands without any robot data.

The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

[Kevin Y Wu @kevin_y_wu]

Introducing Human Universal Grasping (HUG): dexterous grasping learned entirely from human hands, with zero robot data.

🌐 Website: https://t.co/78rfwuuh4J
📄 Paper: https://t.co/BhAI4a1esg
💻 Code: https://t.co/omtjbM7Scl

[Jihan Yang @jihanyang13]

Camera pose matters for video understanding!

Today's MLLMs excel at recognizing activities, but still struggle with the underlying space and ego/object dynamics in video. We trace this gap to a missing piece: camera pose.

Introducing Cambrian-P: a multimodal LLM natively grounded in camera pose. (1/n)

[Irmak Guzey @irmakkguzey]

Learning from human data requires human-like hardware. Humans use their wrists constantly, but table-top manipulators lack this flexibility.

We build upon RUKA and introduce RUKA-v2: a tendon-driven hand with a 2-DOF wrist and finger abduction/adduction 👋✌️

[Jitendra MALIK @JitendraMalikCV]

Teleoperation was pioneered ~1950 to remotely handle radioactive material. When we use it today to collect robot trajectories for BC, it is still clumsy. Surely, there is a better way! (Hint: human video, RL in sim).https://t.co/l7QUge3lwe

[AgileX Robotics @AgilexRobotics]

✨ Meet YOR: Open-Source Bimanual Mobile Manipulator from @nyuniversity

Fully open-source mobile manipulator with dual 6-DoF PiPER arms by AgileX Robotics, BOM cost only ~$10k!

🌐 https://t.co/FksNNYfOgJ
#Robotics #OpenSource #AgileXRobotics #PiPER #NYU

[Siddhant Haldar @haldar_siddhant]

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

https://t.co/0Zi2PUPbE8

[Lerrel Pinto @LerrelPinto]

Introducing YOR.
Balancing budget and functionality for a capable mobile robot is always a challenge. To give researchers and hobbyists more options, we built our own open-source one for ~$10k.

[Mahi Shafiullah 🏠🤖 RSS 2026 ✈️ @notmahi]

Why buy a robot when you can build your own?

Meet YOR, our new open-source bimanual mobile manipulator robot – built for researchers and hackers alike for only ~$10k. 🧵👇

[Jeff Cui @jeffacce]

We don't need the name of an object to pick it up; we simply need to know where it is and what it looks like.
Introducing Contact-Anchored Policies (CAPs): instead of language, we explicitly condition on contacts. Our policy learns object pickup with only 16 hours of data! 🧵

[Mahi Shafiullah 🏠🤖 RSS 2026 ✈️ @notmahi]

Best ideas are often the simplest in hindsight.
Meet Contact-Anchored Policies (CAP)🧢: by conditioning policies on physical contact (vs language) we achieve env & embodiment generalization with super low resources.
This policy ⬇️ learned to pick from scratch w/ 16 hrs of data 🧵

[Irmak Guzey @irmakkguzey]

We just released AINA, a framework for learning robot policies from Aria 2 demos, and are now open-sourcing the code: https://t.co/HSHrtUrt11. It includes:
✅ Aria 2 data processing into 3D observations like shown
✅Training of point-based policies
✅Calibration
Give it a try!

[Irmak Guzey @irmakkguzey]

Dexterous manipulation by directly observing humans - a dream in AI for decades - is hard due to visual and embodiment gaps.
With simple yet powerful hardware - Aria 2 glasses 👓 - and our new work AINA 🪞, we are now one significant step closer to achieving this dream.

[Raunaq Bhirangi @Raunaqmb]

When @anyazorin and @irmakkguzey open-sourced the RUKA Hand (a low-cost robotic hand) earlier this year, people kept asking us how to get one.
Open hardware isn’t as easy to share as code.
So we’re releasing an off-the-shelf RUKA, in collaboration with @WowRobo and @zhazhali01.

[Lerrel Pinto @LerrelPinto]

I gave a Early Career talk at CoRL 2025 in Seoul last week, where I talked about my observations from the past decade in robot learning along with where the field is headed for the next decade.

In summary, the future of robot learning needs:

(1) Data beyond teleop: We are never going to reach the scale of LLM / VLM data by tele-operating robots. Need to leverage consumer hardware already in people's hands (e.g. iPhones) and emerging devices (e.g. Smartglasses).

(2) Observations beyond vision: The hard problem in robotics is dexterity. Dexterity is all about moving objects intricately through contact. The sense of touch is critical for this. Vision can help you acquire objects, but anything more complex will need touch.

(3) Reasoning beyond reactivity: The biggest wins in robot learning have been in reactive policies (both manipulation and locomotion). But the class of models that got us here are generally feed-forward nets. Long-horizon reasoning needs the ability to predict future outcomes and manipulate them. Currently unclear what the right scalable architectures are here, but we are working on it.

(thanks @zacinaction for the pic!)