Steven Oh

Incoming CS PhD @uchicago | MechEng @waseda_univ

[Steven Oh @StevenOh_]

Force sensing for low-cost robot arms — without adding force sensors.

🚀 Excited to share FACTR 2! 🚀
FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning.

w/ @JasonJZLiu  @_tonytao_
🧵(1/6)

[Genesis AI @gs_ai_]

Humanoid robots don't need to look human.

Meet Eno, our first general-purpose robot.

Not a machine pretending to be human, but intelligence given a body.

At Genesis, we’re building a future where robots don’t feel cold or distant, but capable, calm, and ready to help.

Available Q4 this year.

[Steven Oh @StevenOh_]

We have experiments showing how our estimates match the signal from the external torque sensor embedded in each of the joints in a Franka Panda arm. This enabled us to compare known external torque vs our method’s estimate. The results can be found on our website (video with the Franka) and in our paper (Table 1 and Figure 5). You can see how the estimate matches the external torque sensor’s readings, even though the model never saw any “in-contact” data.

[Steven Oh @StevenOh_]

Force sensing for low-cost robot arms — without adding force sensors.

🚀 Excited to share FACTR 2! 🚀
FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning.

w/ @JasonJZLiu  @_tonytao_
🧵(1/6)

[AgileX Robotics @AgilexRobotics]

Force sensing without expensive sensors? 🧠🦾

The team at @CMU_Robotics is pushing boundaries with FACTR 2, unlocking native force awareness on standard commodity hardware with <10 mins of training data.

Website: https://t.co/A8SkAp9x3f
Paper: https://t.co/My6dJ3urBP

[Yangcen Liu @Randle_Liu]

Great work by @StevenOh_ . A lightweight yet effective system introducing tactile information to manipulation.

[Funabashi / 船橋 @funabashihand]

CMUとの共同研究論文が出ました。@StevenOh_ らが、力覚センサを用いないバイラテラル遠隔操作および、接触前の状態を考慮した模倣学習手法を提案しました。結果として、複数のContact-rich manipulationを実現しています。これらの手法は、多指ハンドのマニピュレーションにも応用していきたいと考えています。

[Funabashi / 船橋 @funabashihand]

Excited to share the collaborative research paper with CMU. @StevenOh_ and colleagues worked on bi-lateral tele-operation without force sensors and pre-contact aware imitation learning. They achieved multiple contact-rich manipulation.
I believe the methods can be applied to multi-fingered manipulation.

[Steven Oh @StevenOh_]

@JasonJZLiu @yangphiliphan Sweet

[Jason Liu @JasonJZLiu]

Here are a few more tasks we trained using the system

w/ @yangphiliphan

[Steven Oh @StevenOh_]

@ti_556 Thanks tai but you are the GOAT xd

[Steven Oh @StevenOh_]

@PranavPonni @JasonJZLiu @_tonytao_ Thanks Pranav!

[Chris Paxton @chris_j_paxton]

Contact is the hard part

You can often tell who is faking it based on how much theyre avoiding making sustained contact with stuff

Cool work

[Deepak Pathak @pathak2206]

Force is arguably the most overlooked ingredient in modern robot learning.

Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required.

Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines:

✅ Free force sensing for both the robot and the operator arm
✅ Makes demos higher-quality → fewer of them needed.
✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient.
✅ Strong performance on complex tasks with fewer demos and even no pretraining!

More details below.

[Steven Oh @StevenOh_]

@jeremy_dasa @JasonJZLiu @_tonytao_ No 🥲 i will come visit you in Cornell soon

[Kenny Shaw @kenny__shaw]

Manipulation policies should focus on contact!

FACTR 2 first learns force estimation for any robot arm without requiring any extra sensors.

It uses this to train BC policies that focus on the contact rich moments that matter most for success.

[Russ Salakhutdinov @rsalakhu]

New work on FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning:

Paper: https://t.co/7PXIUrXpAK
Web: https://t.co/IeJeTJ27L8

FACTR 2 shows that learned force signals can both enable force-feedback teleoperation on low-cost manipulators and improve behavior cloning (BC) policies for contact-rich tasks. It consists of two components:

1. Neural External Torque Estimation (NEXT): A lightweight model that infers external joint torques without dedicated force sensors.
2. Force-Informed Re-Sampling Training (FIRST): A training strategy that uses the learned force signal to identify and upsample task-critical moments.

The key insight is simple: policy failures rarely occur in free space, they occur during brief pre-contact alignment and contact-rich interactions, where precise corrections matter most.

Together, NEXT and FIRST bring force-aware teleoperation and robust long-horizon contact-rich policy learning to off-the-shelf robot arms, without requiring additional sensing hardware.

See a more detailed thread by @JasonJZLiu.

[Steven Oh @StevenOh_]

@JasonJZLiu Also check out @_tonytao_ 's thread! https://t.co/D43IRzE1X2

[Tony Tao @_tonytao_]

What if some parts of a robot demonstration are more important than others?

Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact.

In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below.

w/ @StevenOh_ @JasonJZLiu

🧵(1/N)

[Steven Oh @StevenOh_]

Check out @JasonJZLiu's thread https://t.co/mmZQt4a9Q0

[Jason Liu @JasonJZLiu]

💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors.

We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies.

FACTR 2 consists of:
1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors.
2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training.

w/ @StevenOh_ @_tonytao_

🧵(1/N)