EmPRISE Lab

EmPRISE Lab in CS at Cornell University. We are a full-stack robotics lab. Our vision is to EMpower People with Robots and Intelligent Shared Experiences.

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Physical caregiving is one of robotics' hardest frontiers: it is contact-rich, physically intensive, long-horizon, safety-critical, and full of deformable objects.

Physical caregiving tasks such as bathing, dressing, transferring, toileting, and grooming require professional training and considerable practical experience. Yet, no existing dataset captures how expert caregivers perceive, interact, and adapt in real-time when performing these tasks, in a form that robots can learn from.

✨ We introduce OpenRoboCare at #IROS2025, the first expert-collected, multi-task, multimodal dataset for physical robot caregiving, featuring:
🩺 21 expert occupational therapists demonstrating caregiving procedures
🛠️ 15 caregiving tasks across 5 Activities of Daily Living (bathing, dressing, transferring, toileting, grooming)
🧍 2 hospital-grade manikins for safety and repeatability
🎥 5 synchronized sensing modalities: RGB-D, pose tracking, eye gaze, tactile sensing, and expert task & action annotations
📂 315 sessions · 19.8 hrs · 31,185 samples

Beyond raw data, OpenRoboCare distills core physical caregiving insights:
- 3 core principles followed by occupational therapists: pre-positioning, anticipation of body mechanics, and task efficiency.
- 4 key physical techniques: the bridge strategy, segmental rolling, wheelchair recline, and stabilization of key control points.
- Quantitative patterns in task duration, predictive gaze behavior that precedes physical contact, and the timing, magnitude, and spatial distribution of contact forces across body regions and task phases.

The dataset will be made openly accessible through the AWS Open Data Sponsorship Program soon.

🌐 Check out our project website for more visuals and insights: https://t.co/MS3YNEolzQ

This work is led by: @xiaoyul14, @RealZiangLiu, and Kelvin Lin. This is a collaboration with Harold Soh's group from NUS and @DimitropoulouDr from CUIMC.

@EmpriseLab @Cornell_CS @IROS2025 @awscloud

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Robots often rely on whole-arm contact for caregiving tasks such as bed-bathing and transferring. Humans exhibit contact preferences in terms of where and how much force to exert for comfortable interactions. But comfort isn’t one-size-fits-all: preferences vary across people, and for the same person, across different body parts. How can a robot adapt to these preferences on the fly?

Excited to share our work “PrioriTouch: Adapting to User Contact Preferences for Whole-Arm Physical Human-Robot Interaction” led by @rishabhmadan96! #CoRL2025

💡 Core idea
Treat user contact preferences as a ranking over control objectives. PrioriTouch learns this priority ordering online and executes it with Hierarchical Operational Space Control (H-OSC), so higher-priority contacts (the ones closer to causing discomfort) are protected while others yield.

🧠 Learning to rank, safely
We introduce LinUCB-Rank, a contextual bandit that updates the priority ordering from sparse user feedback (“I feel uncomfortable around my abdomen”).

To keep people safe, risky exploration occurs first in a digital twin (simulation-in-the-loop) and then is deployed on the real robot.

📊 What does this buy us?
A sample-efficient way of reasoning about contact preferences. In sim and hardware: fewer force-threshold violations, fewer feedback signals to reach the right ordering, and sustained task efficiency.

@EmpriseLab @ToyotaResearch @Cornell_CS @corl_conf

🗣️ Spotlight: Sep 29 (Session 4)
📊 Poster: Sep 29 (Session 2)

🌐 Website: https://t.co/DKazdptqbR
📝 Paper: https://t.co/a4MRp6hcZp

(1/3) 🧵

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Introducing CLAMP: : a device, dataset, and model that bring large-scale, in-the-wild multimodal haptics to real robots. Haptics / Tactile data is more than just force or surface texture, and capturing this multimodal haptic information can be useful for robot manipulation.

Check out @pranavnnt’s work “CLAMP: Crowdsourcing a LArge-scale in-the-wild haptic dataset with an open-source device for Multimodal robot Perception”, at #CoRL2025.

The CLAMP device is an open-source, low-cost (<$200), portable (0.59 kg) tool that can sense 5 haptic modalities along with vision and language. Users can take it home and log haptic data via a PiTFT screen and buttons.

As far as we know, the CLAMP dataset is the largest multimodal haptic dataset in the robotics literature, with a total of 12.3 million data points from 5357 objects in 41 homes, collected by 16 CLAMP devices.

The CLAMP model is a material recognition model that outperformed GPT-4o, CLIP, and PG-VLM in our experiments, and generalized to haptic data from three different robot embodiments (WidowX and Franka with different grippers).

A finetuned CLAMP model enabled a 7-DoF Franka Panda to robustly perform three real-world manipulation tasks involving clutter, occlusion, and visual ambiguity.

@EmpriseLab @Cornell_CS @corl_conf

🗣️ Spotlight presentation at #CoRL2025 on Sep 30 (spotlight session 5)
📊 Poster session at #CoRL2025 on Sep 30 (poster session 3)
🌐 Website: https://t.co/T4v5JyBiz2
📄 Paper: https://t.co/9OFGgxg3ks

Check this thread for more details (1/6) 🧵

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

During a meal, food may cool down resulting in a change in its physical properties, even though visually it may look the same! How can robots reliably pick up food when it looks the same but feels different — such as steak 🥩getting firmer as it cools? 🍴

Check out @ZhanxinWu0725's work SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition — an oral at #CoRL2025.

SAVOR introduces a novel method to learn skill affordances, which capture how suitable a manipulation skill (e.g., skewering, scooping) is for a utensil–food interaction. Skill affordances arise from the combination of tool affordances (what a utensil can do) and food affordances (what the food allows). Using this method, SAVOR improves bite acquisition success by 13% over state-of-the-art methods.

@EmpriseLab @Cornell_CS @corl_conf

🗣️ Oral presentation at #CoRL2025 — join us on Sep 28 (afternoon session)
🌐 Website: https://t.co/92Pwa5fWMy
📄 Paper: https://t.co/kFaGiNDamw

Check this thread for more details (1/6) 🧵

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

NERC 2025 is happening @Cornell this year. Here is the website with more details: https://t.co/tOClpYYz2G

We have a fantastic set of keynote speakers from a variety of backgrounds @Majumdar_Ani from @Princeton, Victoria Webster-Wood from @CarnegieMellon, @wendyju from @cornell_tech, and @HerlantLaura from RAI

With poster presentations of extended abstracts, Rising Star spotlight talks, and demos / booths and other support from our generous sponsors @FourierRobots @rai_inst @clearpathrobots @UnitreeRobotics @CornellCIS and @CornellCOE , this event is going to be exciting!

Do register (Early Deadline: September 10th, Late Deadline: October 3rd) and come enjoy this event at the beautiful @Cornell Campus in Ithaca on October 11th 🎉🎉

[Rajat Kumar Jenamani @rkjenamani]

Really excited to share that FEAST won the Best Paper Award at #RSS2025!

Huge thanks to everyone who’s shaped this work, from roboticists to care recipients, caregivers, and occupational therapists. ❤️

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

And we won the #RSS 2025 Best Paper Award! Congrats @rkjenamani and the entire @EmpriseLab team @CornellCIS 🎉🎉

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Congrats @rkjenamani and the entire team @EmpriseLab on this impressive accomplishment and being nominated for Best Paper Award and Best Systems Paper Award at #RSS 2025!

This project took almost 2.5 years to get to this stage, and I am incredibly proud of what we have achieved in term of real-world deployment of a meal-assistance system with real users in their homes with minimal researcher intervention leveraging in-the-wild personalization.

Key insight is that for in-the-wild deployment of user-centered systems, adaptation and personalization need to go hand-in-hand with transparency and safety. More technical details are in the thread 🧵below.

@rkjenamani will be presenting this work on Monday (June 23, 2025) @RoboticsSciSys 2025 in the #HRI session. Do attend :-)

Website: https://t.co/YEgvvDhfwj
@CornellCIS @Cornell_CS @EmpriseLab

[Rajat Kumar Jenamani @rkjenamani]

Most assistive robots live in labs.
We want to change that.

FEAST enables care recipients to personalize mealtime assistance in-the-wild, with minimal researcher intervention across diverse in-home scenarios.

🏆 Outstanding Paper &amp; Systems Paper Finalist @RoboticsSciSys
🧵1/8

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

So proud of what we achieved with the PhyRC Challenge at #ICRA2025 with the support of our generous sponsors @KinovaRobotics and @hellorobotinc! From simulation to real-world assistive care tasks -- robot-assisted dressing and bed-bathing -- this was an ambitious undertaking. Huge congrats to all the teams who won a Stretch 3 robot (sponsored by @hellorobotinc) and a Gen 3 robot (sponsored by @KinovaRobotics) as prizes 👏. The teams were impressive but these tasks are extremely challenging, and no team was able to completely finish the tasks. Want to give these tasks a try during the next iteration of #PhyRC? Stay tuned for the next version :-) @EmpriseLab @CornellCIS

[Rajat Kumar Jenamani @rkjenamani]

Excited to share our work on continual, flexible, active, and safe robot personalization w/ @tomssilver, @RealZiangLiu, Ben Dodson &amp; @TapoBhat.

Also: @tomssilver is starting a lab at Princeton!! I HIGHLY recommend joining — thoughtful, kind, and an absolute joy to work with!

[Tom Silver @tomssilver]

Happy to share a new preprint: "Coloring Between the Lines: Personalization in the Null Space of Planning Constraints" w/ @rkjenamani, @RealZiangLiu, Ben Dodson, and @TapoBhat.

TLDR: We propose a method for continual, flexible, active, and safe robot personalization.

Links 👇

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Congrats @rohanbbanerjee and the team on getting nominated for Best paper award at #ICRA2025 for their work on “To ask or not to ask: Human-in-the-loop contextual bandits with applications in robot-assisted feeding”. Check out his presentation in Room 302 on Tuesday at the Awards Finalists 2 session at 11:15am.

In robot-assisted feeding (and many other real-world tasks in unstructured settings), full robot autonomy can be fragile as algorithms and systems may fail. Can a robot leverage a user, who is already present, for help vs. acting autonomously?

In our new work, we introduce LinUCB-QG: A human-in-the-loop contextual bandit algorithm that adapts querying based on both task uncertainty and a predicted user-specific workload based on a querying model learned using a newly collected diverse dataset.

Key takeaway is that users with mobility limitations may experience higher workload when queried compared to users without mobility limitations, so the robot queries less, prioritizing lower querying workload even at some cost to task performance. However, user satisfaction is high. Our system improves bite acquisition success while reducing user workload — validated via simulation and real-world user studies. This work was also done with @rkjenamani, Sid Vasudev, Amal Nanavati, @DimitropoulouDr, and Sarah Dean.

Website: https://t.co/AhcPTcvKt0

@EmpriseLab @CornellCIS @Cornell_CS @ieee_ras_icra @ieeeras

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Congratulations to @EmpriseLab for winning the Unity for Humanity Grant Award ( https://t.co/6BYSUhNL7B ) with our RCareWorld project.

All credit goes to @Nekovowo for leading this project!

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

How can caregiving robots adapt their control policies across people with diverse mobility limitations, balancing robot assistance with user agency in action across various tasks?

Check out @RealZiangLiu et al.'s work @EmpriseLab
@CornellCIS: GRACE: Generalizing Robot-Assisted Caregiving with User Functionality Embeddings, where we represent a user’s mobility using learned functional embeddings to enable personalized, adaptive robot assistance. This was presented @HRI_Conference
last week.

We first collect DataGRACE, an open-source dataset of functional range of motion using motion capture data and functional scores administered by occupational therapists across 11 users simulating 4 different medical conditions using occupational therapy practices with resistive bands.

We use DataGRACE to develop our algorithm GRACE, which learns functional embeddings for a user’s mobility representation and uses this to adapt downstream robot control policies for people with diverse mobility limitations.

Using four simulated tasks —handover, rehab, dressing, bed-bathing, and a real-world handover task, we evaluate GRACE’s ability to generalize caregiving policies across diverse users. GRACE significantly improved user agency in action and task success compared to relevant heuristics based baselines.

This work is a collaborative effort with Prof. Katherine Dimitropoulou, an Occupational Therapist by training, from Columbia University Irving Medical Center (CUIMC).

Website: https://t.co/PbRtjkYeKa

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

How can a caregiving robot with limited torque help people with mobility limitations and older adults transfer from a hospital bed to their wheelchair?

Check out @Nekovowo's work at HRI @HRI_Conference: CART-MPC: Coordinating Assistive Devices for Robot-Assisted Transferring with Multi-Agent Model Predictive Control, where we took a first step by enabling a robot to transfer a manikin from a bed to a wheelchair while coordinating with assistive devices that exist in people's homes. @EmpriseLab @CornellCIS @Cornell_CS

Website: https://t.co/JQc22D6oaR

Check this thread for more details (1/7) 🧵

[HRI Pioneers @HRIPioneers]

Welcome #HRIPioneers2025! @rkjenamani from Cornell University will present their work 'Towards Deployable Physical Caregiving Robots: A Case Study in Mealtime Assistance' at @HRI_Conference

Read more on Rajat's website: https://t.co/Kg3P7H70mN

[Rajat Kumar Jenamani @rkjenamani]

🚨 Call for Papers: Physical Caregiving Robots Workshop 🚨
📍 HRI 2025 | Melbourne, Australia 🌏🦘

📄 Submit a 2-page extended abstract on topics related to Physical Caregiving Robots

🔗 Key Links:
Website: https://t.co/YuwHwgORIN
Register interest: https://t.co/VZKvprlWQh

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

We finalized our Phase 2 (Real-world Phase) venue! Our competition proposal was accepted at ICRA 2025 in Atlanta🎉 @ieee_ras_icra

Here’s the timeline:
🗓️ Phase 1 ends: Dec 23, 2024
🗓️ Phase 2 participants announced: Dec 30, 2024

So far, 56 teams are participating, tackling:
Track 1: Fixed-base Manipulation for Robot-assisted Dressing
Track 2: Mobile Manipulation for Robot-assisted Bed Bathing

Top teams from Phase 1 will advance to compete in Phase 2 with real robot prizes sponsored by @KinovaRobotics and @hellorobotinc.

🌟 Huge thanks to everyone contributing to this challenge—let’s keep the momentum going! We’re thrilled to see participants from around the globe using our simulation platform.

If you haven’t joined yet, there’s still time—registration is open, and it’s not too late to be part of this exciting journey! We have regular office hours to help you with your questions.🚀

🔗 Learn more: https://t.co/hPD3x9UYVJ

#ICRA2025 #Robotics #PhyRC

[Tapomayukh "Tapo" Bhattacharjee @TapoBhat]

Cornell opened it's interdisciplinary Robotics Ph.D. program: https://t.co/DoMTXVuNZg

Do apply: https://t.co/DxOpKmLdPh

Deadlines:
Dec 1 for AE and ME
Dec 15 for CS and ECE

@Cornell @Cornell_CS @CornellEng @HadasKressGazit