Intelligent Robotics and Vision Lab @ UTDallas

Welcome to the X page of the Intelligent Robotics and Vision Lab @UT_Dallas

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

Many #robot_learning works use human videos but need lots of data/retraining.

We present #HRT1 — a robot learns from just one human video and performs mobile manipulation tasks in new environments with relocated objects — via trajectory transfer.🔗 https://t.co/zbOGWrSHAF (1/11)

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

Show the robot a task once, and it does the rest 🤖

That's #HRT1, which @saihaneesh_allu and I showcased at #AIDays — teaching a mobile manipulator a task from a single human demo video.

Nice to see our lab featured on the @UTDJonsson site afterward.

Credit to the @IRVLUTD team and our advisor Dr. @YuXiang_IRVL.

🔗Feature: https://t.co/iDaGOkgR1k
🔗HRT1: https://t.co/AZA64BxuM3

@UT_Dallas
@UTDResearch
@UTDallasAlumni
@UTDallasNews

#Robotics #RobotLearning #RobotPerception #UTDallas

[Jiahui Zhang @JiahuiZhang__32]

Real-world VLA evaluation is still hard to standardize.

We introduce VLA-REPLICA: a low-cost, reproducible benchmark for evaluating VLA models.
By standardizing hardware setup and scene initialization, it can be easily reproduced across research labs and institutions.

🧵1/10

[Yu Xiang @YuXiang_IRVL]

It was a pleasure to host a high school robotics team from Rockwall, TX in our lab @IRVLUTD yesterday and share some of our latest robot demos.

Thanks to @saihaneesh_allu , @lfcasas7 and @JiahuiZhang__32 for helping organize and showcase the demos!

[Yu Xiang @YuXiang_IRVL]

It was great to have @jinwanxin visiting us @IRVLUTD today!

Dr. Jin presented ComFree-Sim, a fast contact physics simulation framework, and shared exciting work it enables in in-hand manipulation and physics-aware perception. Very impressive work!

[Yu Xiang @YuXiang_IRVL]

We (@lfcasas7 and I) handed out 21 NVIDIA #Jetson Orin Nano kits to students in my computer vision class this semester.

All teams showcased their live Jetson-powered demos in the classroom at the end of the course!
Here are a few sample projects👇
1. Hand gesture for control

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

RT @jishnu_jaykumar: 🤖 From live robot demos in a new space, to meeting inspiring researchers, winning Best Poster 🏆, and being featured he…

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

Had the pleasure of meeting @rao2z during #ECSAIDays2026 at our demo booth.

I’ve been following his work since my undergraduate days, so this was a genuinely special moment. Really appreciated the time he took to stop by, engage with our demo, and share thoughts.

Moments like these make the long hours behind the scenes feel worth it.

@IRVLUTD
@UT_Dallas
@UTDCompSci
@UTDResearch

#Robotics #AI #Research

[Sai Haneesh Allu @saihaneesh_allu]

Excited to show our HRT1 framework live today at #ECSAIDays2026.

If you're curious how robots mimic human actions, especially outside controlled setups, we’ll be running real demos and sharing some of the practical challenges behind it.

Visit us at:
📍 ECS West 2.100, UT Dallas
🗓️ April 30 from 11 AM

@UT_Dallas @IRVLUTD

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

Stepping out of the lab and into the real world.

Today at #ECSAIDays2026, we’ll be presenting a live demo of #HRT1 — a system that transfers human demonstrations to robot actions for mobile manipulation. 🔗 https://t.co/AZA64BxuM3

This short timelapse is a behind-the-scenes glimpse of what it actually takes.

From moving the robot across campus, navigating buildings, setting up hardware, testing repeatedly, to making everything work outside controlled environments — a lot goes into what eventually looks like a “simple demo.” Especially in smaller labs, it’s all hands-on, end-to-end effort.

Also, when it’s just two of us running a two-person setup and trying to film it, the camera doesn’t always cooperate 😄 — but the live demo will be much more fun.

Grateful to be building this with @saihaneesh_allu at the @IRVLUTD

If you’re around, drop by and see it live.
📍 ECSW, UT Dallas
⏳ Apr 30, 2026

Thanks to @Sriraam_UTD, @VibhavGogate, @YuXiang_IRVL and Tyler Summers for the opportunity and support.

@UT_Dallas
@UTDCompSci
@UTDResearch

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

🛠️ While working on #iTeach (https://t.co/UVv9EsM0nJ), where unseen object instance segmentation (UOIS) was a key task, I kept hitting the same wall: every UOIS dataset had its own loader and quirks. Getting data ready for training meant writing custom glue for each — time that should have gone into the models themselves.

✨ So I put together uois_toolkit — a small PyTorch library that wraps 5 popular UOIS datasets (Tabletop, OCID, OSD, Robot Pushing, iTeach-HumanPlay) behind one API.

🚀 Features:
 📦 Load any dataset in 3 lines
 📊 Compute F1, IoU, Precision, and Recall with a single call
 ⚡ Plug directly into PyTorch Lightning
 🤖 Works out of the box with robotics pipelines

💡 The idea is simple — take the friction out of data pipelines so more time can go into model building.

🎉 It has picked up around 2K downloads on PyPI since release, which was a nice surprise. 🙏 Sharing it a bit more openly now in case others find it helpful.

❤️ Huge thanks to the original dataset authors, whose open codebases made this possible. And to Avaya Aggarwal and Animesh Maheshwari for testing and feedback along the way.

🔗 GitHub: https://t.co/Rvvq7WMFnb
📥 PyPI: https://t.co/wk4GTMYTq3

@IRVLUTD @UT_Dallas

#Robotics #RobotPerception #ComputerVision #PyTorch #Lightning #OpenSource

[Yu Xiang @YuXiang_IRVL]

Great to have @Jesse_Y_Zhang visiting us @IRVLUTD today!
He shared his journey toward generalist robotics reward models (RoboCLIP, ReWiND, Robometer), followed by a great buffet with the lab.

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

🤖 Robots don't fail in the lab. They fail in the wild — clutter, occlusion, constantly changing environments.

The real question: Can robots learn directly from these failures during deployment?

How about teaching robots the way we'd teach a child — by showing them where they went wrong?

🧵👇

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

✨Honored to receive the Louis Beecherl, Jr. Graduate #Fellowship (2026–2027) from the Erik Jonsson School of Engineering and Computer Science at 🟠@UT_Dallas

🙌Grateful for the support as I continue my #PhD research in #RobotPerception and #Learning at the @IRVLUTD | @UTDResearch

[Jishnu Jaykumar Padalunkal @jishnu_jaykumar]

A day well spent with @IRVLUTD at #TEROS26! Presented #HRT1, and had some great idea exchanges with fellow roboticists 🤖✨

[Yu Xiang @YuXiang_IRVL]

A group photo in the end @IRVLUTD

[Yu Xiang @YuXiang_IRVL]

We noticed that DINOv3 was surprisingly strong at matching object features.

This inspired L2G (Local Matches to Global Masks). With a few reference images, a robot can search a room for the target object.
🔗 Project: https://t.co/PTy0C8xlTS
💻 Code: https://t.co/QWRP5IBEd3

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

We’re excited to see Robometer out!
A pleasure to collaborate on scaling general-purpose robotic reward models to 1M+ trajectories.
Looking forward to seeing how the community builds on this!

[Jesse Zhang @Jesse_Y_Zhang]

A reward model that works, zero-shot, across robots, tasks, and scenes?

Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.

Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!

🧵 (1/12)

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

Congratulations to Felipe!!

[Luis Felipe Casas @lfcasas7]

Thrilled to have won the Louis Beecherl, Jr. Graduate Fellowship for 2025–2026 from the Erik Jonsson School at @UT_Dallas !

This prestigious merit-based award has been a game-changer for my PhD in Computer Science and research at the Intelligent Robotics and Vision Lab—real financial support and recognition.

Grateful to the committee!

Current/future UTD grad students: Apply for the 2026-2027 Graduate fellowships! It eases the journey and rewards excellence. Don't miss out. 🚀

#UTDallas #Fellowship

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

An excellent resource for ROS 2 beginners!

[Yu Xiang @YuXiang_IRVL]

If you want to learn ROS2 with MoveIt2 and Gazebo simulation, check out the five homework in my robotics course:
https://t.co/s0vS2xXXgg

By the end, you'll have a Fetch robot picking up a cracker box. Everything runs in Docker on Windows or Mac. #robotics #ROS2

[Intelligent Robotics and Vision Lab @ UTDallas @IRVLUTD]

We celebrated Thanksgiving early at an Asian buffet last week. Happy Thanksgiving everyone! 🦃