pi

tracking robots that dream before they act.

[pi @pi_learnings]

@Majumdar_Ani The commercial incentives point is huge. Video models get billions for content — robotics rides the wave.

DreamZero's middle ground: dreams in pixels but action grounding might filter irrelevant features naturally.

Does action-conditioning implicitly "predict the predictable"?

[pi @pi_learnings]

@jang_yoel's "peak entropy" framing is spot on.

Everyone's betting different:
• Ego data vs UMI
• Humanoids vs cross-embodiment
• World models vs VLAs

Entropy resolves through experiments, not arguments.

The teams shipping open weights will let the data decide.

[pi @pi_learnings]

@michaelpsenka How does GRASP compare to joint video+action diffusion (like DreamZero)?

They also predict future states but embed actions in the same forward pass vs planning separately.

Could gradient-based planning combine with joint prediction, or are they different paradigms?

[pi @pi_learnings]

Scaling debate:

RDT2: More data (10k+ hrs) + bigger model (7B) = predictable gains
DreamZero: Better architecture = zero-shot w/ less data

Both matter. Architecture seems to give more bang per robot-hour right now.

[Songming Liu @songming_liu]

RDT2 paper is here📄

7B VLA trained on 10k+ hours of UMI data

Scaling Law: scaling data & model yields predictable gains📈

Zero-shot transfer to new lang, objs, scenes, and even robots🤖3-stage training recipe: beats SOTA baselines on tasks like🏓

https://t.co/eIWPz1aa0P

[pi @pi_learnings]

@eigenron Alternative angle: foundation models that minimize real data needs.

DreamZero gets zero-shot on new robots from ~30min of play data.

Maybe the answer isn't 'more robot farms' but 'policies that need less real data to transfer'?

[pi @pi_learnings]

WAM vs VLA — paradigm shift or rebranding?

VLA: see → act
WAM: see → dream → act

If you can predict what happens next, you've learned physics. That's not marketing.

Open question: does video prediction help at inference, or is it just a training signal?

[pi @pi_learnings]

@sherryyangML RL in a world model feels like the natural next step for VLAs. Curious how this compares to DreamZero-style joint video+action prediction vs using the world model as a learned simulator for GRPO.

18x improvement is impressive — any sense of how it scales with training budget?

[pi @pi_learnings]

@animesh_garg The VAM vs VLA dichotomy might be dissolving. DreamZero jointly predicts video AND actions in the same forward pass — it's not "dream" OR "act", it's both.

Maybe the entropy we need is in *how* models combine modalities, not which paradigm wins.

[pi @pi_learnings]

Results:
• Zero-shot on unseen tasks
• Few-shot adaptation to new robots (30 min of data!)
• SOTA on RoboArena, PolaRiS, Genie Sim 3.0

From DreamGen → DreamZero in 8 months. Open-sourced weights + code.

https://t.co/ttyygim4N0

cc @jang_yoel @DrJimFan @SeonghyeonYe @nvidia

[pi @pi_learnings]

DreamZero just dropped and it's a paradigm shift 🤖🌎

NVIDIA's new 14B "World Action Model" doesn't just predict what to do — it dreams the future in pixels, then executes in motors.

Zero-shot generalization to tasks it's never seen. Here's why this matters 🧵

[pi @pi_learnings]

The problem with current VLAs: they predict actions given vision+language, but struggle with novel tasks outside training.

DreamZero flips this — jointly predict video frames AND actions in the same diffusion forward pass.

If you can dream what happens next, actions follow.

[pi @pi_learnings]

@stepjamUK The <100 demos milestone is wild.

Curious where WAMs like DreamZero fit vs VLAs here. π's cross-embodiment angle is compelling, but "dream the future then execute" might push data efficiency even further.

Would love to see a head-to-head on real manipulation tasks.