Nick Stracke ✈️ CVPR

PhD Student at Ommer Lab (Stable Diffusion)
Working on video analysis and world understanding...

[Nick Stracke ✈️ CVPR @rmsnorm]

Video diffusion models learn motion indirectly through pixels.

But motion itself is much lower-dimensional.

We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics.

This enables efficient planning -> 10,000× faster than video models.

🧵👇

[jo.schb ✈️CVPR @jo_schb]

⚠️ Standard first stages are not sufficient for safety-critical applications!

The most extreme weather events are often the hardest to decode.

One latent → many plausible reconstructions

Deterministic decoders hide that uncertainty.

Meet FREUD 🧵👇

[Nick Stracke ✈️ CVPR @rmsnorm]

Check out our work on how to scale NVS on internet-scale data! We provide fixes to the unsupervised NVS pipeline (RayZer) and also obtain more interpretable pose estimations while simplifying the overall setup.

[Stefan Baumann✈️CVPR🔜SF @StefanABaumann]

The internet is full of video. So why can't novel view synthesis just scale on it?

Real-world video is simultaneously unposed, messy, and dynamic, breaking self-supervised NVS.

We fixed that. RayDer learns static-scene NVS from dynamic internet video, scaling like an LLM. A🧵

[Nick Stracke ✈️ CVPR @rmsnorm]

💡 Training with differently noised patches increases overall image gen performance, as the model learns a better underlying representation.

This holds even for plain Euler sampling, but their sampler increases the gap even more!

[jo.schb ✈️CVPR @jo_schb]

Diffusion models treat every part of an image equally.
→ Same number of steps. Same compute.

But images aren’t uniform. 🤔

Some regions are easy, others are hard.

So why force the model to treat them the same? 🧵

[Nick Stracke ✈️ CVPR @rmsnorm]

@jonchu dm

[Nick Stracke ✈️ CVPR @rmsnorm]

@nurvai_ai You need periodic regrounding, and that's also what we do for LIBERO. You usually also have a translation error from converting tracks into actions that a robot can actually execute, which you also have to compensate for.

[Nick Stracke ✈️ CVPR @rmsnorm]

Video diffusion models learn motion indirectly through pixels.

But motion itself is much lower-dimensional.

We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics.

This enables efficient planning -> 10,000× faster than video models.

🧵👇

[Simo Ryu @cloneofsimo]

Cool stuff

[Nick Stracke ✈️ CVPR @rmsnorm]

Stop predicting motion step-by-step. Model the whole motion in a compact representation for efficient planning.

📄 Paper: https://t.co/S51t6kxMqY
💻 Models: https://t.co/MohDxwjpz0

Joint work with @KoljaBauer, @StefanABaumann, @itsbautistam, Josh Susskind, and Björn Ommer.

[Nick Stracke ✈️ CVPR @rmsnorm]

@itsbautistam @KoljaBauer @CVPR Thanks Miguel!

[Miguel Angel Bautista @itsbautistam]

Amazing work led by @rmsnorm @KoljaBauer and our collaborators at LMU, to be presented at @CVPR! Personally, I find this question of "what's the right level of abstraction for planning in physical space?" to be very intriguing. Pixels over time are very low SNR (ie. the argument behind JEPA) but motion/trajectories carries a lot on information while being extremely compressible. I believe there's a lot more to uncover from this direction. Very glad to be part of this one!

[Brian Roemmele @BrianRoemmele]

A massive step forward for AI video!

[Nick Stracke ✈️ CVPR @rmsnorm]

@Frid45 Thanks! 🇪🇺 Supporting European research, I see 👀

[atharva ☆ @k7agar]

I have been saying

[Nick Stracke ✈️ CVPR @rmsnorm]

@KoljaBauer @StefanABaumann @itsbautistam 1️⃣https://t.co/2wWMnibvRa
Also, shoutout to two other recent works that explore how to use point tracks for world modeling.
👇...

[Nick Stracke ✈️ CVPR @rmsnorm]

@KoljaBauer @StefanABaumann @itsbautistam 2️⃣https://t.co/OyDxtn4kzg

[Stefan Baumann✈️CVPR🔜SF @StefanABaumann]

You don't imagine the future by mentally rendering a movie. You trace how things move -- abstractly, sparsely, step by step.
We built a model that does exactly this. It predicts motion, not pixels -- and it's 3,000× faster than video world models.
Myriad, accepted at @CVPR 2026

[Nick Stracke ✈️ CVPR @rmsnorm]

@KoljaBauer @StefanABaumann @itsbautistam 1️⃣https://t.co/2wWMnibvRa
Also, shoutout to two other recent works that explore how to use point tracks for world modeling.
👇...

[Neerja Thakkar @neerjathakkar]

What’s the right representation for a world model? 3D, pixels, or something else?

Excited to release our new paper “Forecasting Motion in the Wild” where we propose point tracks as tokens for generating complex non-rigid motion and behavior

From @GoogleDeepmind @Berkeley_AI @TTIC_Connect

[Kolja Bauer @KoljaBauer]

Do we really need pixel generation to model motion? 🤔

We show how directly representing motion in a compact space enables efficient, scalable planning.

10,000× faster than video models, enabling planning and reasoning in open-world and robotics settings.

Check it out ⬇️