Sicheng Li

SteerNeRF, NeRFCodec, GSCodec_Studio. PhD student @ ZJU

[Sicheng Li @JasonLsc22]

@pablovelagomez1 @krahets @rerundotio really interesting, especially the turbo LoRA part. looking forward to more when you put it out :)

[HU Wenbo @wbhu_cuhk]

No game engine. No 3D reconstruction.

Just a still image and a camera path we picked in advance. The model walks it. S-curves, orbits, paths that double back on themselves.

What we changed is how the model knows where anything is. A video transformer only gets a sense of space from its positional encoding, and that encoding basically says "row 12, column 30, frame 7." That's where a token sits in the tensor. It says nothing about where it is in the world.

So now each token carries the camera ray it's looking along instead. A ray belongs to the camera, not to whatever it's pointed at, so you know it before you've generated a single pixel.

Once the geometry is in the coordinates you get 3D consistency for free. Loop the path back and you come home to the same room you left.

One addition into q and k. Under 0.1% new params. On a 14B backbone: 29% lower camera rotation error, 43% lower FVD.

Code and weights are up, Apache-2.0:
https://t.co/hAdFrUeliS
https://t.co/laIIrbYhyg

#WorldModels #VideoGeneration

[Zhening Huang @ZheningHuang]

Introducing LiteReality-Agent 🏠🤖✨

An agentic system that turns room scans into realistic, interactable 3D scenes.

No tool currently exists that can turn an entire room scan into an interactable 3D scene, so we built one! 🚀

Scan your room with the app we built, then let LiteReality-Agent turn it into a cool digital copy with articulated assets. 🛋️✨

Have fun! 🚀

🌐 Project page: https://t.co/8SpAtf1YAX
💻 Code: https://t.co/mxBxGO59C6
📱 App: https://t.co/yqcsJEFVPL
📝 Blog: https://t.co/8ffR3PmTAG

[Sicheng Li @JasonLsc22]

Solid

[Hanwen Jiang @hanwenjiang1]

Interested in how frontier labs pre-train image/video generation models?
We were too.

Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.

In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.

Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.

This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.

But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.

The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.

Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.

A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch

[Sicheng Li @JasonLsc22]

@RadianceFields MPEG have been working on the compression of radiance field for a long time

[Alexi Gladstone @AlexiGlad]

We discovered a third pretraining axis beyond parameters and data: exploration.

Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation.

In the simplest case, it's just a for loop.

Introducing Explorative Modeling.

TLDR:
- Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute
- Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet
- Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is
- End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute

🧵Thread:

[ali @waterloo_intern]

https://t.co/aivc7B4A7Z

[Hanming Ye @DozenDucc]

Introducing Waddle Labs: Claude Code for robots.

Connect our API to your robot and enter a prompt, then our agents write code to achieve the task in 20 minutes.

@yiding_song @theWaddleLabs

[Sicheng Li @JasonLsc22]

@DylanTFWang Quite useful data engine

[InSpatio｜影溯 @InSpatio_AI]

A video is compressed 4D data. 🌍

Today, we’re sharing a preview of our latest research: 4D reconstruction from a single monocular video. 🎬

From one casually captured video, our system reconstructs a dynamic, freely viewable 4D Gaussian scene—recovering geometry, appearance, camera motion, and object motion across space and time.

No multi-camera rig. Just video → 4DGS.

The world has already been captured in video. The next step is to unlock the 4D structure inside it.

Watch the result below. 👇

#4DGS #GaussianSplatting #CVPR #CVPR2026 #AIGC #AI #3DGS

[Francesco Sacco @FrancescoSacco1]

New high quality blog post: "How to train a Frontier-Level world model" in collaboration with @reactorworld

This is going to be very useful for practitioners

[Massimiliano Viola @massiviola01]

Image foundation models never stop surprising!😮

And this time, it's not the usual big tech players.

A few days ago, @robbyant_brain released LingBot-Vision, a new family of vision encoders built natively for dense spatial perception.

[NaN Xue @NanXue7]

See the depth prediction for robot manipulation on the most challenging case: TRANSPARENT objects! A plastic bag pouring beans into glass jars, and the perceived depth stays consistent over time, with clean and sharp BOUNDARIES.

Why is the depth this good? The hero behind the LingBot-Depth 2.0 upgrade is LingBot-Vision, our new self-supervised vision pretraining that makes boundary geometry the learning signal of SSL. The model discovers its own boundary tokens from raw images (no labels, no edge detector, bottom-left panel) and forces exactly those into the mask, so the student must reconstruct boundary geometry from context. Bottom-right: a single frozen-feature query staying locked on the deforming bag.

[Michael Black @Michael_J_Black]

The term “world model” is tossed around a lot but this is the real deal. From video and actions, the model learns a consistent representation of the physics, 3D environment, and game state of Rocket League.

I'm excited to have been part of this project because I see this as a step towards learning world models that can generalized to the real world. I’m a believer in learning from synthetic data (Sintel, SURREAL, AGORA, BEDLAM, BEDLAM2) and I’ve seen how models trained on synthetic data can generalize.

MIRA is an important step because it’s learned from video gameplay generated completely by bots. This provides a pathway to scaling and scaling is key to real-world generality. Traditional graphics and games provide a path to learning rich models of the world.

Check out the detailed tech report: https://t.co/C9aJPY8XRX

[Hao AI Lab @haoailab]

https://t.co/lwO6dSHm9m

[Sicheng Li @JasonLsc22]

@YeheLiu amazing speed

[Zian Wang @zianwang97]

🚀 What if physical AI policies could interact with generated worlds in real time?

Introducing OmniDreams, a generative world model for closed-loop autonomous vehicle simulation.

Tech report, code, models, and data samples are available now.

Project: https://t.co/BOTWdSJKMx
Code: https://t.co/hPH3KbE6Uy
Model: https://t.co/G4g9TWFD2W

Join the #omnidreams discord channel: https://t.co/AIwYQvc0bv

[Ruilong Li @ruilong_li]

World models are moving beyond offline generation towards interactive, real-time experiences.

Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models:

🔥 Up to 3.10× faster LingBot-World inference
🔥 Up to 2.12× faster Self-Forcing inference
🔥 Up to 1.40× faster Wan2.1 inference
🔥 8 integrated models
🔥 Multi-GPU, streaming, low-latency serving
🔥 Agentic skills that teach you how to use it

FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds.

Github: https://t.co/xM8LuPaRTS
Docs: https://t.co/IInORNIzy3
Research page: https://t.co/mZ6TLQSpIO
Join the #flashdreams Discord channel at https://t.co/GGOQ0k7liY

FlashDreams is also the runtime backbone behind NVIDIA OmniDreams (https://t.co/PLUt55gxxh)

1/n

#AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

[Nataniel Ruiz @natanielruizg]

this makes a ton of sense
we argued for decoupling visual generation of world models and the "game engine" in some of our works (multigen and unbounded)
you can track/simulate in many different ways and the visual model doesn't have to do all the heavy lifting