Siyuan Cen

CVer

[Ben ✈️ CoRL 2026 @bentropy_ai]

Astra made robot control look like a job for frontier LLMs.

But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need.

Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵
(1/11)

[Yaqi Xie @xie_yaqi]

Very excited to share that I'll be joining UIUC ECE and CS (affiliate) as an Assistant Professor @UofIllinois

I'm recruiting PhD students, interns, and postdocs to build trustworthy AI agents and robots that reason, learn, and grow alongside humans. If you're interested in joining my lab at UIUC, reach out at [email protected]

Many thanks to my advisors and collaborators for all their help and support! @haroldsoh @ Katia_Sycara @RamananDeva @ Alexander_Gray @ David_Hsu @ksmeel @ Mohan_Kankanhalli @joecampb @SimonStepputtis @ Woojun_Kim @yswhynot

[Zhiqiu Lin @ZhiqiuLin]

Before AI can generate professional videos, it needs to see like a professional.

We spent a year with 100+ content creators teaching AI to describe video like a filmmaker would.

Introducing CHAI: Critique-based Human-AI Oversight for Building a Precise Video Language [CVPR'26 Highlight, Top 3%].

Try prompting a video generator for a dolly zoom, dutch angle, point of view, or camera roll. Most fall back to the same bland defaults: a push-in, a level shot, a third-person view. Why? These techniques require a language of cinema that current models rarely speak.

We built that language:

1️⃣ Precise specification: 5-aspect structured captions co-designed with professional cinematographers covering subject, scene, motion, spatial, and camera dynamics

2️⃣ Scalable oversight: LLMs draft captions, humans critique what's wrong and how to fix it

3️⃣ Post-training recipes: Qwen3-VL-8B surpasses Gemini-3.1 and GPT-5

4️⃣ Video generation: fine-tuned Wan follows 400-word cinematic prompts with precise control

Here's how each works 🧵

Work led by CMU and Harvard with @chancharikm, @du_yilun, and @RamananDeva.

📄 Paper: https://t.co/wCwEtvrntM
🌐 Site: https://t.co/oAAQklGrfF

[Vincent Sitzmann @vincesitzmann]

In my recent blog post, I argue that "vision" is only well-defined as part of perception-action loops, and that the conventional view of computer vision - mapping imagery to intermediate representations (3D, flow, segmentation...) is about to go away.

https://t.co/aFmE9CHHau

[Boyuan Chen @BoyuanChen0]

Introducing Large Video Planner (LVP-14B) — a robot foundation model that actually generalizes. LVP is built on video gen, not VLA. As my final work at @MIT, LVP has all its eval tasks proposed by third parties as a maximum stress test, but it excels!🤗
https://t.co/wjD54YFK3k

[Siyuan Cen @censiyuan1]

@chancharikm Really cool work! Congrats!

[Siyuan Cen @censiyuan1]

@gan_chuang Totally agree, industry attracts more talent and resources. Research today is highly collaborative, and no one works in isolation anymore.

[Chuang Gan @gan_chuang]

As I see more of what my students are building at frontier robotics startups, I’m increasingly convinced industry will soon dominate robotics learning, just as it has with LLMs and image/video generative models, overtaking academia.

[Chancharik Mitra @chancharikm]

🎉Despite massive pretraining, VLAs need to adapt to specific physical contexts. We introduce Robotic Steering, a novel finetuning method using mechanistic interpretability to surpass standard FT:

🎁 22× fewer parameters
🎁 +53% on unseen tasks
🎁 Interpretable

Thread below👇

[Zihan Wang @Z1hanW]

Introduce CRISP, a real-to-sim pipeline that recovers human motion and simulatable scene geometry from monocular video!

CRISP builds contact-faithful 3D scene for simulation - 8× fewer sim failures, +43% faster sim, and improves human motion!

Interactive demos👉: https://t.co/locrdrxO16

Exciting collaboration w/ @JiashunWang @jefftan969 @_Tsukasane @ Jessica Hodgins @shubhtuls @RamananDeva

[Zhiqiu Lin @ZhiqiuLin]

🎉CameraBench has been accepted as a Spotlight (3%) @ NeurIPS 2025. Huge congrats to all collaborators at CMU, MIT-IBM, UMass, Harvard, and Adobe.

CameraBench is a large-scale effort that pushes video-language models to reason about the language of camera motion just like professional cinematographers.

🌍 Our open-source dataset, models, and code are also gaining strong interest and adoption from frontier labs such as DeepMind and Kling to advance video generation research.

📄Paper: https://t.co/Rn6pArwdlv
🌐 Website: https://t.co/RnVQfzSiUc

[Siyuan Cen @censiyuan1]

It’s impressive to see reasoning capabilities emerging in video models.

[Luma @LumaLabsAI]

This is Ray3. The world’s first reasoning video model, and the first to generate studio-grade HDR. Now with an all-new Draft Mode for rapid iteration in creative workflows, and state of the art physics and consistency. Available now for free in Dream Machine.

[Siyuan Cen @censiyuan1]

Thanks for AK reporting our work! More than excited to continue research in detailed and comprehensive video understanding!

[AK @_akhaliq]

Towards Understanding Camera Motions in Any Video is out on Hugging Face

[Gabriele Berton @gabriberton]

Every single VLM/MLLM paper in the last few months uses Qwen as LLM

Not Gemma, not Llama, not DeepSeek

But Qwen

Every single one

[Zhiqiu Lin @ZhiqiuLin]

📷 Can AI understand camera motion like a cinematographer?

Meet CameraBench: a large-scale, expert-annotated dataset for understanding camera motion geometry (e.g., trajectories) and semantics (e.g., scene contexts) in any video – films, games, drone shots, vlogs, etc. Links below!

We contribute a taxonomy of motion primitives, co-designed over months with professional cinematographers, and apply rigorous quality control to label and caption all aspects of camera motion.

CameraBench shows that even the best SfMs and VLMs struggle with real-world, dynamic videos. Yet, a generative VLM post-trained on our high-quality data matches SOTA SfM (MegaSAM) in geometric understanding and outperforms SOTA VLMs (Gemini-2.5 / GPT-4o) in semantic understanding, e.g., describing how the camera moves.

📄 Paper: https://t.co/DozKB5sRF1
🌐 Website: https://t.co/RnVQfzSiUc

Work led by CMU, MIT-IBM, UMass, Adobe, Harvard, Emerson with @censiyuan1, @chancharikm, @JayKarhade, @du_yilun, @gan_chuang, and @RamananDeva.