Optics as code - a new paradigm in physical AI
Optical neural networks are notoriously hard to calibrate and suffer from the sim2real gap. We show that a coding agent with the hardware in the loop quickly tunes the optics and improves image classification from 35% to 96.8%.
🧵1/6
The same harness also trains an optical image encoder, and more capable agents improve results.
The agent only needs a way to set a device's parameters and a way to observe what it does, so this can be used on any physical system that can be controlled and measured.
5/6
Our new work asks: does a diffusion transformer need a separate decoder? DWD reads pixels directly from its final features while preserving the denoising trajectory. The surprising part: this separation also improves quality for direct pixel generators.
Latent diffusion needs no decoder😵
New paper: DWD, a linear head on DiT predicts pixels directly
• Same or better quality on every tested latent model, less flicker in Wan video
• Even pixel models improve: ImageNet FID 1.55 ➡️ 1.43 (pixel SOTA), photorealistic texture in T2I
@HanshengCh The pixel-model ablation is super interesting: the same perceptual losses hurt FID with a shared head but improve it with separate heads. Denoising and final image formation benefit from different objectives, even when both operate in pixel space.
What should a flow model predict? Let it learn. Our new work, AnisoFlow, learns a direction-dependent prediction target while keeping the architecture, loss, and sampler unchanged. Better image quality and remarkably clean pixel-space generation—even in 8-bit.
What should a pixel flow model predict? Data, velocity, or others?
New paper: AnisoFlow🔥
• A fully learnable anisotropic target
• Theory that explains what it learns
Results:
• ImageNet: FID 1.90 → 1.55 over JiT, no REPA
• Pixel T2I without patch artifacts, even in 8-bit
@HanshengCh The learned target aligning with the data covariance is especially revealing: the choice between data and velocity prediction depends on direction. Nice to see the theory explain what the model learns—and the same formulation address patch artifacts.
Introducing pi-H3⚡, the fastest MiniMax H3 yet. 10 s of 768p video with audio generated in 9.2 s on a single B200.
How we did it:
Direct pixel output in 2 or 4 steps, no video VAE decoder.
Blog: https://t.co/z9LJ7cWW7t
App: https://t.co/haaESsn6nN
@StanfordAILab@LumaLabsAI
@HanshengCh@StanfordAILab@LumaLabsAI What excites me most is how much survives at just 2 steps: rich textures and dynamic motion. Generating this in less time than it takes to watch makes trying another idea—or ten—a very different creative experience.
Rhoda will be at #CoRL2026 in Austin 🤖
🍸 Tue Nov 10, 6:30–8:30 PM: Video Models Meet Robotics reception at Swift's Attic. Invite-only, request a spot: https://t.co/xIXYGuSu2f
🎤 Thu Nov 12, 11:25 AM: Vincent Sitzmann @vincesitzmann speaks at the "Do Robots Need World Models?" workshop
https://t.co/8cxdELBfoM
What excites me most about mobile DVA is how little additional robot data it takes to learn robust mobility and manipulation. A powerful example of how web-scale video pretraining can make new robot capabilities easy to learn.
We taught a robot how to move around in only 15 minutes.
That’s all the mobility data our Direct Video-Action-powered robot needed to accomplish a task that requires mobility and manipulation.
@RhodaAI High success rates with so little additional robot data—that’s the striking result here, especially when generalizing to unfamiliar layouts and unseen obstacles.
I'd like to advertise this paper in advance of CoRL 2026 where you have the opportunity to check it out. Yet another example of the power of human videos as a data source, and the cool thing is that one can augment real videos with synthesized variants, produced with the help of a video generatuon model. The real2sim2real paradigm rules!
Robots rarely fail on the clean runs. They fail on the edge cases.
On Field Trips, @schrep and @gigascale watched our robot hit one live on camera and work through it on its own, with no human stepping in.
That's what out DVA model is built for.
🎥 https://t.co/xq21Kfe8VW
A robot that only works when everything goes to plan isn't autonomous. It just hasn't met the real world yet.
During @schrep visit, our robot ended up with a bag upside down on the wrong side, a case it never trained on. It figured it out with the power of @RhodaAI's DVA model.
Excited about this new direction! Tokenizers typically optimize reconstruction (VAEs) or discrimination (DINO), creating a mismatch with downstream generation. RePR introduces a principled, scalable approach to learning representations specifically as tokenizers for generation.
1/ The tokenizer is a new scaling axis!
We introduce an approach for diffusable representation learning that's principled, scalable, and works when DINO/LeJEPA collapse.
We get joint tokenizer+diffusion scaling laws and it's compute-optimal vs raw diffusion
w/ @drfeifei@GordonWetzstein@jcjohnss@ericryanchan@vincesitzmann@KyleSargentAI
Thread, Paper, and Blogpost below
@BaadeAlan@drfeifei The watermark experiment shows RePR overcoming a key weakness of LeJEPA: an objective that can be satisfied by a shortcut while useful image structure is lost. RePR continues to learn meaningful features even when the watermark makes image discrimination trivial.