Tai Wang

Research Scientist at Shanghai AI Lab. Embodied AI & Spatial Intelligence.

[Wenzhe Cai @WenzheC7616]

🤖Can robots achieve accurate navigation without any external localization feedback?
📸We present #LoGoPlanner, which handles perception, localization, and planning in one go!

Check our results on LeKiWi, G1, and Go2 robots.
🌐Project: https://t.co/mWIMzIfuqT

[Runsen Xu @runsen_xu]

1/3
Introducing MMSI-Video-Bench, the most challenging Spatial Intelligence benchmark. Even the strongest MLLM, Gemini 3 Pro, scores only 38%, revealing a ~60% human–AI gap. 🤯

🌐 Project Page: https://t.co/mvNQ2AL9QB
💻 GitHub: https://t.co/WocFXjifmp
🤗 Hugging Face: https://t.co/8ncUfy1lFv
📄 arXiv: https://t.co/nZ4yNGi43m

Benchmark Features
1️⃣ Challenging: all evaluated models struggle, with large and persistent human–AI performance gaps.

2️⃣ Diverse scenarios: 25 public + 1 in-house datasets, covering egocentric, third-person, indoor, outdoor, driving, aerial, and movie scenes.

3️⃣ High quality: fully human-annotated by 11 3D vision experts under strict quality control.

4️⃣ Comprehensive tasks: spatial layout reasoning, motion understanding, planning, prediction, and cross-video reasoning.

5️⃣ Domain-oriented sub-benchmarks: Indoor Scene Perception Bench, Robot Bench, and Grounding Bench for targeted evaluation.

Experiments
1️⃣ We evaluate 25 models. Most achieve low scores, with an average ~60% human-AI gap.

2️⃣Models struggle significantly with motion, planning, prediction, and cross-video reasoning.

3️⃣Prediction is the most challenging main category, and camera–instance spatial relations are the hardest subtype.

4️⃣Notably, spatially fine-tuned models fail to generalize to MMSI-Video-Bench.

🥳🥳 Also, check out our previous MMSI-Bench, another challenging benchmark for multi-image spatial intelligence.

Project Page: https://t.co/0ldu40p2Gr

[LeRobot @LeRobotHF]

🚀 Introducing X-VLA ; LeRobot’s new soft-prompted Vision-Language-Action model.

X-VLA is built to scale across many embodiments: different robots, cameras, action spaces, and environments, all handled by one unified transformer backbone.

- Generalist across robots (Franka, WidowX, Agibot, sim + real)
- Soft-prompt domain IDs let the model adapt to new hardware with tiny learnable embeddings
- Flow-matching + transformer core for smooth, continuous 50 Hz control
- Pretrained on a mixed-embodiment dataset spanning 7+ platforms and diverse tasks
- Fine-tune on any dataset using one of the 6 checkpoints we provide out of the box.

[Wenbo Hu @gordonhu608]

🚀 Introducing G^2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning

G^2VLM can natively predicts 3D attributes (depth, camera pose, pointmaps) and uses them for spatial understanding via interleaved reasoning.

🔧 End-to-End Unified Model
✅ Monocular & Video Depth Estimation
✅ Pose Estimation
✅ 3D Point Reconstruction
✅ Spatial Understanding and Reasoning

🏆 This design helps G^2VLM achieve robust performance in both 3D Reconstruction and Spatial Reasoning!

#VLM #spatial #3D #LLM
💻 Code at: https://t.co/WxvTgeG9Jw
👇[1/n]

[Jinliang Zheng @2_toinf]

🚀 New: X-VLA (0.9B)-A Lightweight VLA model for cross-embodiment manipulation
🧺 2+ hrs cloth folding autonomy, trained with only 1.5K demos
📊 Benchmarks: 96% Simpler-WidowX, 97.4% Libero-Long, 70% RoboTwin, + more
Project Page: https://t.co/Jq3e66ypQs
(1/N)

[Tai Wang @wangtai97]

Excited to release InternVLA-N1 officially with a little delay, after maybe my last attempt to post it on arXiv but failed🫥

Project page (w/ tech report): https://t.co/Tq2QjOt6dA

It features a dual-system framework that decouples high-level spatial planning from low-level execution, enabled by a two-stage curriculum training paradigm and a large-scale dataset, InternData-N1.

It achieves SoTA performance in both simulation and real-world evaluation and has great zero-shot sim2real capabilities.

We also released the corresponding toolkit at https://t.co/8XoVJyn14e, which includes training and evaluation support for the entire system, covering both conventional benchmarks from Habitat and newly built ones from Isaac Sim.

Welcome any feedback on our dataset and codebase, especially zero-shot testing with our pretrained models (InternVLA-N1, StreamVLN & NavDP) 😇

BTW, it is really disappointing to release a work without arXiv and HuggingFace @huggingface daily paper🥹. I still do not know the exact reason why my paper was rejected, and there is no way to communicate🥹. @arxiv

[Tai Wang @wangtai97]

🙌🏻Welcome to try and join our challenge! Looking forward to strong submissions winning the great award🫡😋

[Intern Robotics @InternRobotics]

🚀 3 steps to ace IROS 2025 Nav Track: Setup · Develop · Submit 🦾
📺 We’ve prepared a Quickstart Guide to help you quickly grasp the task, explore the dataset, and submit your model to the leaderboard.
🥇 Winner prize: $10K 📌 https://t.co/xEIqoH2rbY

[Intern Robotics @InternRobotics]

🚀 Intern Robotics just open-sourced a wave of HOT datasets on @huggingface — from navigation to manipulation, interaction & motion!
Built for embodied AI, robotics, and multimodal research.#EmbodiedAI #opensourceai
👉Explore all: https://t.co/T0VIEK0yQN

[AK @_akhaliq]

StreamVLN

Streaming Vision-and-Language Navigation via SlowFast Context Modeling

[Wenzhe Cai @WenzheC7616]

🤖Can we build a generalized robot navigation policy without any real-robot data?

👏We introduce the NavDP, which can zero-shot adapt to different robots in the open world.

Website: https://t.co/jH7rNZpe1v
Github: https://t.co/UEBDazgKbJ
Arxiv: https://t.co/qu4x4ls1de

[Yixin Chen @_yixinchen]

📢📢📢Excited to announce the 5th Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics at #CVPR2025! Expect our awesome speakers and challenges on multi-modal 3D scene understanding and reasoning. 🎉🎉🎉@CVPR

Learn more at https://t.co/hZEQr5qKxu.

[Elgce @BenQingwei]

🫰Thrilled to introduce HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit.

Website: https://t.co/ZjcSUifB47
Code: https://t.co/lNv6gydur0
YouTube: https://t.co/eMX25eVwPH

😀 HOMIE consists of a novel RL-based training framework and a self-designed hardware system. You can find their core features here:

🥳 1. RL framework:
a. Without any motion prior, it helps different humanoid robots to balance under any changing upper-body poses.
b. It makes the robot capable of squatting to any given heights robustly, thus enlarging the workspace.

🏋️‍♂️ 2. Hardware System:
a. Its isomorphic exoskeleton design makes it possible to set joint angles directly and read values at over 200 Hz, getting rid of errors introduced by IK.
b. Using HOMIE, one single operator can control the full body of a humanoid, performing diverse loco-manipulation in complex scenes.

✊ Notably, we introduce HOMIE to GRUtopia, making it possible to teleoperate robots both in the real world and the simulation.

🙂 HOMIE is fully open-sourced, which means you can DIY your own cockpit (without commercial purposes.)

🙌 HOMIE can be used as:
1. A codebase for further RL research.
2. A solid device for you to efficiently collect loco-manipulation demonstrations.
3. A reliable humanoid cockpit for you to directly drive your robot!!!

😁 We introduce the detailed technical points in our paper, if you want to have a more comprehensive of HOMIE, please check it out~

[Jiangmiao Pang @pangjiangmiao]

Excited to introduce the Perceptive Internal Model (PIM) for Humanoid Robots!

The first policy simultaneously for:
- Go up and down stairs, jump gaps, and 50cm high platforms.
- Indoor and outdoor scenarios.
- Unitree H1 and Fourier GR-1 robots.

Paper: https://t.co/x1gq0XTBEc

[Runsen Xu @runsen_xu]

How to achieve 3D perception without reconstructed point clouds or additional training, using only generalizable 2D and language foundation models?

At #CoRL2024, we introduce VLM-Grounder, a zero-shot VLM agent for 3D visual grounding.

Paper: https://t.co/hOVaHgkLA6 with codes.

[Winston Gao @WinstonGu_]

Imagine a future where you can ask humanoid robots to clean your room, but some items, like heavy sofas, are too challenging for just one robot to move.

Introducing CooHOI, a learning-based framework designed for the cooperative transportation of objects by multiple humanoid robots. 🤖🤼🤖

Our work has been accepted as Spotlight at NeurIPS 2024. Website: https://t.co/gWFSYEqAAD

[Tai Wang @wangtai97]

@runsen_xu @junting9 hhhhhhhh six!

[AK @_akhaliq]

LLaVA-3D

A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness

Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.