Lingfeng Sun

Building robots | PhD @UCBerkeley | previous: Researcher @rai_inst

[Lingfeng Sun @ling_feng_sun]

What can GPT-6 Astra do on a humanoid when the contact gets hard?

Give it the right interaction kernel.

Led by @yk20wang, our team presents KPI: A Promptable Kernel for Physical Interaction on Humanoids.

With our agentic system, Astra completes long-horizon, contact-rich tasks zero-shot—no task demos or task-specific training.

1/6

[Lingfeng Sun @ling_feng_sun]

@_ming_qin in practice we also use external timecode generator https://t.co/IEEk15pCm3

[Lingfeng Sun @ling_feng_sun]

We introduce KIWI: a modular robot-data rig built around Insta360 cameras.

Voice-controlled capture. No carried computer or trackers.

Ego/wrist video, training-ready motion and a 3D scene—from the same recording.

1/5

[Yikai Wang @yk20wang]

Check out KPI: a general real-time interaction kernel on top of a humanoid whole-body tracker, unlocking diverse contact-rich loco-manipulation while preserving general-purpose tracking. Pair it with a powerful agent like GPT-6 Astra to turn high-level instructions into physical interaction!

[Lingfeng Sun @ling_feng_sun]

KPI: a physical interaction kernel between multimodal models like GPT-6 Astra and robot controllers.

Full team: @yk20wang, @ZhuAdrien, @huxiao93612565, Hao Zhang, Zelin Wang, @VSyeungggg, @zhao__ding & @ling_feng_sun.

Paper + videos: https://t.co/u4WOlD6jZo

6/6

[Lingfeng Sun @ling_feng_sun]

What can GPT-6 Astra do on a humanoid when the contact gets hard?

Give it the right interaction kernel.

Led by @yk20wang, our team presents KPI: A Promptable Kernel for Physical Interaction on Humanoids.

With our agentic system, Astra completes long-horizon, contact-rich tasks zero-shot—no task demos or task-specific training.

1/6

[Lingfeng Sun @ling_feng_sun]

Beyond the agentic system, we show open-loop planned trajectories to write and wipe on a vertical board, and teleoperation examples for more whole-body tasks: kneel, stand, turn and walk while carrying a box, and open a drawer against resistance.

KPI closes the loop on contact and can be used in different systems.

More videos on the website!

5/6

[Lingfeng Sun @ling_feng_sun]

@3DPrintAficio This is exact the "first principle" I have in mind in this project (we had the chance to add "X" to make "Y" better). We are trying (and hope the community can help) to maximize the performance under this principle, and that also drives us to write this report and share.

[Lingfeng Sun @ling_feng_sun]

@Vikashplus yes, insta360 has a webcam mode for online use.

[Lingfeng Sun @ling_feng_sun]

@omgsdgomg Definitely worth exploring! For KIWI, we prioritize simplicity and scalability, cameras are all you need. The question is whether LiDAR’s accuracy and robustness gains justify the extra hardware and integration. I’m actively testing that for outdoor collection.

[Lingfeng Sun @ling_feng_sun]

@menkhofflukas Cloud upload is optional, we record locally to SD cards and swap cards/batteries as needed. Offloading and processing happen offline, so collection doesn’t have to wait for uploads.

[Lingfeng Sun @ling_feng_sun]

Insta360 cameras are great data collection devices. high quality recordings, microphone, storage, voice control come with the product. My team actually reproduced how only two x5 cameras can reproduce high quality UMI data in a recent project. We initially made the same “AprilTag” guess but later found out it's might not a necessary component after some iteration😆. https://t.co/R2LUH8tznz

[Lingfeng Sun @ling_feng_sun]

We introduce KIWI: a modular robot-data rig built around Insta360 cameras.

Voice-controlled capture. No carried computer or trackers.

Ego/wrist video, training-ready motion and a 3D scene—from the same recording.

1/5

[Lingfeng Sun @ling_feng_sun]

Team: Benjamin C. Yang, Weiying Wang, Shenggao Li, Keming Yan, Sasha Wilkinson, Zelin Wang, Yip Fun Yeung & Lingfeng Sun.

Project: https://t.co/n5JKNLKBJm
Report: https://t.co/E0ZuIT4hEV

5/5

[Lingfeng Sun @ling_feng_sun]

The microphones are sensors, too.

KIWI uses shared sounds in the cameras' own audio to align their recordings offline, bringing both wrists and an optional head-camera view onto one timeline.

4/5

[Lingfeng Sun @ling_feng_sun]

New work with @SichangSu42538 : real-world RL for long-horizon manipulation. online RL only on the bottleneck subtasks; agents write the selector, verifier and reset, rollouts need no human corrections. Humans reset only what the robot "cannot".

[Sichang Su (Destiny) @SichangSu42538]

Your fine-tuned VLA does most of a long-horizon task, then gets stuck at a few critical steps.

Don't retrain it. Don't collect more demos of what already works.

Let it practice the part that fails, on the real robot.

PARTS, w/ @ling_feng_sun (project lead) 🧵

1/8

[Lingfeng Sun @ling_feng_sun]

Come and show your agents!

[Haoru Xue @HaoruXue]

🚨 Call for live demo / paper at CoRL 2026 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗪𝗼𝗿𝗸𝘀𝗵𝗼𝗽! No paper required for demo track. Due Sep 27.

🔮 The real magical power of agent is live interactive demo with the audience, which is why we provide all-out support:

- Robot, compute, and API are ready for you in the conference room

- Custom robot / sim demo also welcomed

- Just submit a video proof - friendly to industrial participants

https://t.co/F2L9Q05PAK

Paper submission also welcomed!

We also have an amazing speaker lineup from CMU, UC Berkeley, DeepMind, NVIDIA, and Tencent.