Ruochen Xu

Engineer @GalbotRobotics. PhD graduate from CMU LTI @LTIatCMU. Working on embodied AI and agent.

[Tiancheng Zhao (Tony) @tianchezhao]

Visual Navigation & Following for Everyone.

OpenTrackVLA is a fully open-source Vision-Language-Action (VLA) stack that turns monocular video and natural-language instructions into actionable, short-horizon waypoints. Happy Tracking!

GitHub: https://t.co/dqNdhsHiqe

[Tiancheng Zhao (Tony) @tianchezhao]

Latest blog on teaching VLMs to understand fine-grained objects.

VLM-FO-1 equips a novel object-enhanced vision tower, achieving remarkable object understanding performance with only 3B parameters. Larger models and RL-enhanced coming soon.

https://t.co/2rQJbzw2Pe

[Ruochen Xu @RuochenXu1]

2) Results?
+28.8% 0-shot accuracy on GSM8K (Llama3-8B)
Outperforms Magpie & prompt engineering 🚀
Preserves general skills (no catastrophic forgetting!)

[Ruochen Xu @RuochenXu1]

Recent works show promise with ""RL with only one training example"" or "Absolute Zero" data but with an external environment for code execution. We take it further: self-improving LLMs without any external data and environment!

[Tiancheng Zhao (Tony) @tianchezhao]

🧠 Can LLMs truly self-improve? Enhancing their reasoning ability using only their own outputs, with zero external data or reward models?

Introducing CRESCENT, a minimal, fully auto pipeline for generating math QA data. #ACL2025

📄 Paper: https://t.co/XeUYZXzTE0

[Ruochen Xu @RuochenXu1]

1) How
🪤 Bait prompts to generate math questions
🧩 Self-deduplication (no repetitive Qs!)
🗳️ Majority voting for confident answers

[Ruochen Xu @RuochenXu1]

Qwen2.5VL is great at objection detection (OD). And our RL recipe can make it even better with an interesting "aha" moment for OD.

[Tiancheng Zhao (Tony) @tianchezhao]

1/3: 🚀 Thrilled to share VLM-R1’s latest results! After hitting SoTA in REC & Math, we’ve supercharged RL for open vocab detection (OVD).

TL;DR: With the right rewards, RL-powered VLM nails SoTA on OVD + sparks cool "aha" moments.

Dive in: https://t.co/ipBRtRadSi

[OmAI Lab @OmAI_lab]

🚀 We just dropped a new RL fine-tuned VLM ranking #1 on Open Compass Multimodal Math Benchmark (<4B params)! 🏆

✨ New features:
• Multi-image input 🖼️🖼️
• Customizable base models ⚙️

🔥 Check it out:
https://t.co/XZd9o647f5

#AI #MachineLearning #OpenSource

[OmAI Lab @OmAI_lab]

🌟 VLM-R1 just got SUPERCHARGED!🚀

🔥 Multi-Node Training for GRPO: Scale training across clusters! Tackle massive vision-language tasks 2x faster with our new multinode_training_demo.sh script.

🎛️ Fine-Grained Parameter Control: Tweak num_iterations for high-precision tasks🎯! Balance exploration vs. exploitation with epsilon—stabilize training & boost generalization!

Level up your VLMs NOW😍 #VisionLanguage #DeepSeek #GRPO

https://t.co/ylW2ceikPz

[Tiancheng Zhao (Tony) @tianchezhao]

Introducing VLM-R1!

GRPO has helped DeepSeek R1 to learn reasoning. Can it also help VLMs perform stronger for general computer vision tasks?

Our preliminary answer is YES and it generalizes better than SFT.
https://t.co/iffweRXcpO

[Tiancheng Zhao (Tony) @tianchezhao]

Introducing VLM-R1!

GRPO has helped DeepSeek R1 to learn reasoning. Can it also help VLMs perform stronger for general computer vision tasks?

The answer is YES and it generalizes better than SFT.

https://t.co/iffweRXcpO

[Ruochen Xu @RuochenXu1]

GRPO can help vision-language tasks like Referring Expression Comprehension (REC)

📊 Key insight:

R1 training > SFT: Steady improvement on OOD data, while SFT degrades with more steps.

https://t.co/SPH0KxfjZB

#AI #MachineLearning #VisionLanguage #R1 #OpenSource

[Tiancheng Zhao (Tony) @tianchezhao]

In EMNLP, I attended a great agent tutorial hosted by @ysu_nlp @Diyi_Yang @ShunyuYao12 , learned a lot about language agents and the progress in agentic ops, e.g. ReAct, CoT etc. How do these methods perform given the same model? I did a study to find out! https://t.co/Rb3ONO69mG

[Adina Yakup @AdinaYakup]

Came across some amazing work by @OmAI_lab! 🔥

✨ Open Agent Leaderboard: Track the best AI agents in one place!
https://t.co/Zv8xX90XDj

✨ OmAgent: A multimodal agent framework for video understanding, handles everything from CCTV to full-length films 🎥
https://t.co/Dm2kVodXtZ

[OmAI Lab @OmAI_lab]

The new OmAgent framework is a game-changer.

I just connected my phone to create an AI-powered agent that can detect objects, analyze scenes, and provide instant feedback.
All in under 5 minutes—no coding required. It’s that simple! 🚀

#OmAgent #AI #SmartDevices

[Ruochen Xu @RuochenXu1]

@OmAI_lab Can you show how the deployed agent looks like?

[Jiao Sun @sunjiao123sun_]

Mitigating racial bias from LLMs is a lot easier than removing it from humans!

Can’t believe this happened at the best AI conference @NeurIPSConf

We have ethical reviews for authors, but missed it for invited speakers? 😡

[Ruochen Xu @RuochenXu1]

Paper: https://t.co/PV1twADXsj
Code: https://t.co/o50TKElCEX

[Ruochen Xu @RuochenXu1]

Check out our newest work- ZoomEye! It brings true vision reasoning to VLMs. With a straightforward yet powerful tree search algorithm, it enables any VLM to conduct human-like zoom in/zoom out searches on high-res images.

[Tiancheng Zhao (Tony) @tianchezhao]

This is probably one of the most interesting work that I have seen in a while about enabling visual reasoning for VLMs. Even I am one of the authors, I am still amazed by its simplicity and effectiveness.
https://t.co/M1j7myiHSz

[OmAI Lab @OmAI_lab]

Struggling to pick your OOTD every morning?😫Do the clothes you choose not match the weather? 🌧️☀️

Why not let OmAgent handle it? Get suggestions based on the day’s weather from your location and time, choose the best options in your real-time video and recommend matching pieces!

👕✨ Say goodbye to “What do I wear today?” and hello to effortless mornings.
📱 Easy to use. 💸 Completely FREE.

Why not try the demo at OmAgent and make your own agent!🔗 https://t.co/kod8UlXjyI

#AI #SmartLife #OOTD #Opensource #OmAgent #OmAI

[Ruochen Xu @RuochenXu1]

@thoma_gu @CIS_Penn @PennEngineers Congratulations, Jiatao!