xintong hu

PHD at @XLangNLP in @HKUniversity, advised by @taoyds.
undergraduate from Zhejiang university
Research field：Embodied AI, VLA, WAM

[xintong hu @Erics_Tong]

Current robot policies overfit specific language templates, handling 'pick and place' but freezing on 'drag it to me ' or 'push it closer to me.' They also lack control over execution: which hand, what approach angle, where to grasp, which path to follow.

🤖 FineVLA make robots steerable : changing instruction alters execution; same task, different phrasing, distinct actions — all faithfully done.

🏠 Homepage: https://t.co/NVRV63Wweb
📄 Paper: https://t.co/RUsR51k688
💻Codebase: https://t.co/DyrjoFV395

🧵[1/6]

[XLANG NLP Lab @XLangNLP]

Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved?

Not even close.

🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks.

What's new:
🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human
⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0
🌍 Grounded in authentic artifacts & stateful user profiles
⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more

📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use.

🏠 Homepage: https://t.co/tudMC0pFwC
📄 Paper: https://t.co/17PDTOiR1t
💻 Code: https://t.co/L13F8CoqgN
🤗 Dataset: https://t.co/L1HHHygFk7

🧵 [1/8]

[xintong hu @Erics_Tong]

@taoooo917 @cai_xu77719 thanks, Yitao!

[xintong hu @Erics_Tong]

Current robot policies overfit specific language templates, handling 'pick and place' but freezing on 'drag it to me ' or 'push it closer to me.' They also lack control over execution: which hand, what approach angle, where to grasp, which path to follow.

🤖 FineVLA make robots steerable : changing instruction alters execution; same task, different phrasing, distinct actions — all faithfully done.

🏠 Homepage: https://t.co/NVRV63Wweb
📄 Paper: https://t.co/RUsR51k688
💻Codebase: https://t.co/DyrjoFV395

🧵[1/6]

[xintong hu @Erics_Tong]

🧵[6/6]

FineVLA makes robot policies steerable by specifying how to act, not just what to do.
We open-source the full stack:
✅ Fine-grained data annotation pipeline
✅ RoboFine-VLM annotator
✅ RoboFine-Bench benchmark
🏠 Homepage: https://t.co/NVRV63Wweb
📄 Paper: https://t.co/RUsR51k688
💻Codebase: https://t.co/DyrjoFV395
🤗Annotator: https://t.co/JB8yEThpNk
🤗Benchmark : https://t.co/mHBmM1aLx6

[xintong hu @Erics_Tong]

Current robot policies overfit specific language templates, handling 'pick and place' but freezing on 'drag it to me ' or 'push it closer to me.' They also lack control over execution: which hand, what approach angle, where to grasp, which path to follow.

🤖 FineVLA make robots steerable : changing instruction alters execution; same task, different phrasing, distinct actions — all faithfully done.

🏠 Homepage: https://t.co/NVRV63Wweb
📄 Paper: https://t.co/RUsR51k688
💻Codebase: https://t.co/DyrjoFV395

🧵[1/6]

[xintong hu @Erics_Tong]

🧵[5/6] Key findings:
🔬 i. No Sacrifice — Fine-grained data doesn't hurt goal-level success. FG-only consistently outperforms Raw-only by +1.4 to +8.1 pts. The OFT-vs-GR00T architecture gap shrinks from 6.4 to just 0.8 — showing strong cross-architecture generalization.

🔬 ii. Complementary — FG and raw instructions are complementary, not competing. Performance follows a clear inverted-U, peaking at FG:Raw = 1:1. Best mix: 86.8%/82.5% in simulation(+15/+11.1 over Raw-only ), 62.7 in real-world (+12.8 over Raw-only).

🔬 iii. Steerable — Fine-grained language gives robots true factor-level controllability. Same task, different instructions → different execution:
- Object pose: 24 → 47 (+23)
- Approach direction: 60 → 78 (+18)
- Target color: 22 → 40 (+18)
- Rotation: 76 → 86 (+10)

[Bowen Wang @BowenWangNLP]

RLVR has become the recipe for agentic post-training. But for Computer-Use Agents, the bottleneck is not the algorithm, it is the data. 🐌

🚀 We introduce CUA-Gym: a scalable, lightweight synthesis engine that turns arbitrary task queries into verifiable RLVR data for computer-use agents. The largest open CUA RLVR dataset to date:

🎯 32,122 verifiable RLVR tasks with programmatic setup scripts + rewards
🌐 110 environments: 16 desktop apps + 94 synthesized mock web apps
🏆 Qwen3.5-based CUA models trained with GSPO reach 72.6% on OSWorld-Verified and 56.6% on WebArena

📄 Paper: https://t.co/cdvHJPzgb1
🏠 Homepage: https://t.co/kvhaOQxNx7
🤗 Dataset: https://t.co/w5vOIRdchR
💻 Codebase: https://t.co/CcRlNTlS1c
🧩 Environments: https://t.co/fNZ6YAI8LD
🧵[1/6]

[xintong hu @Erics_Tong]

@OWW Thank you for your support. Please feel free to communicate with us if you have any questions.👏

[xintong hu @Erics_Tong]

Super stoked to be part of Qwen-vla! So glad to meet such team member who share the same interests.

[Shuai Bai @shuai_bai_]

Excited to share Qwen-VLA paper, our exploration of generalist Vision-Language-Action models.
It extends Qwen’s multimodal backbone from visual understanding and reasoning to continuous action generation and trajectory prediction.
Paper:
https://t.co/9jvRW0nI8B

[Shuai Bai @shuai_bai_]

Excited to share Qwen-VLA paper, our exploration of generalist Vision-Language-Action models.
It extends Qwen’s multimodal backbone from visual understanding and reasoning to continuous action generation and trajectory prediction.
Paper:
https://t.co/9jvRW0nI8B