Zizhang Li

CS PhD @Stanford, Prev: @ZJU_China, 3D Vision

[Zizhang Li @zizhang_li]

@tarat_211 Currently it’s being rendered into image plane as the video model 3D rope support only x, y and t coordinates

[Zizhang Li @zizhang_li]

🌍 Actions that condition world models are highly heterogeneous: forces on objects, camera movements, hand motions, robot arms, each with its own meaning and structure. So today's action-conditioned world models are usually built one action type at a time, each with its own conditioning design.

We introduce UniAction, a unified action-conditioning framework: one world model driven by all of these actions, and even by mixes of them in a single generation. Trained jointly, it outperforms the action-specific experts. Great work led by amazing @BowenXue_ .

More detailed threads below👇
🧵 1/10

[Zizhang Li @zizhang_li]

@Cortixofficial Forces are from the simulated data, we take the data @GillmanLab created for his GoalForce work

[Zizhang Li @zizhang_li]

🌍 Actions that condition world models are highly heterogeneous: forces on objects, camera movements, hand motions, robot arms, each with its own meaning and structure. So today's action-conditioned world models are usually built one action type at a time, each with its own conditioning design.

We introduce UniAction, a unified action-conditioning framework: one world model driven by all of these actions, and even by mixes of them in a single generation. Trained jointly, it outperforms the action-specific experts. Great work led by amazing @BowenXue_ .

More detailed threads below👇
🧵 1/10

[Zizhang Li @zizhang_li]

PS: @BowenXue_ is applying to PhD programs this year and he is really great!

[Zizhang Li @zizhang_li]

🌄 And UniAction carries these actions into scenes far beyond its training data.

We've open-sourced the model, data and weights all together. Check out the project page for more details.

Project: https://t.co/cDdWqvFnDj
Paper: https://t.co/XdA4jvtLEz
Code: https://t.co/WjXiQOEBcQ

Joint work with co-first author @BowenXue_, @Koven_Yu and @jiajunwu_cs at @StanfordSVL @StanfordAILab.

🧵 10/10

[Zizhang Li @zizhang_li]

@ericmchen1 @justachetan amazing work!

[Zizhang Li @zizhang_li]

🤯A GREAT benchmark led by @ruihong04, @zzigakovacic, @Pkulits .

We saw tons of real-to-sim results from coding agent these days, this benchmark systematically evaluate the capabilities, from a 4D inverse graphics perspective!

The study features 18 coding agents on a series of real-world and synthetic scenes with diverse, complex physics. 💥🌊💨🪢

Please check out the webpage: https://t.co/O7q8yEsGbo. We show A LOT OF quantitiative and qualitative results and analysis.

[Žiga Kovačič @zzigakovacic]

Do coding agents understand the dynamics of the world well enough to reconstruct it from video?

We introduce 4DCodeBench to evaluate this ability through 4D inverse graphics.

https://t.co/QlkzTPZpbi
<🧵>

[Žiga Kovačič @zzigakovacic]

How do agents model dynamics?
We analyze agents' design choices and find their approaches differ substantially. Across solutions, 67% describe motion analytically, 19% implement custom simulations, 10% use Blender physics, and 3% rely on keyframing.

Opus 5.5 simulates in 76% of its solutions, while Astra does so far less often, with 13% at Low, 22% at High, and 32% at Max reasoning. [5/n]

[Zizhang Li @zizhang_li]

🤯A GREAT benchmark led by @ruihong04, @zzigakovacic, @Pkulits .

We saw tons of real-to-sim results from coding agent these days, this benchmark systematically evaluate the capabilities, from a 4D inverse graphics perspective!

The study features 18 coding agents on a series of real-world and synthetic scenes with diverse, complex physics. 💥🌊💨🪢

Please check out the webpage: https://t.co/O7q8yEsGbo. We show A LOT OF quantitiative and qualitative results and analysis.

[Žiga Kovačič @zzigakovacic]

Do coding agents understand the dynamics of the world well enough to reconstruct it from video?

We introduce 4DCodeBench to evaluate this ability through 4D inverse graphics.

https://t.co/QlkzTPZpbi
<🧵>

[Zizhang Li @zizhang_li]

@AlexHe00880585 JostPapers when