Introducing PointZeroβa 3D world model pre-trained without robots.
Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it?
PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics!
Our model achieves SOTA results on:
π₯ Zero-shot 3D dynamics
π₯ Action-conditioned 3D dynamics (post-training)
π₯ Imitation learning (post-training)
Details and links π 1/6
@y_m_asano@GoogleDeepMind Awesome work, thanks for contributing to the open-source community! Consider citing/benchmarking with our (also fully open) work on Modality Forcing with FLUX: https://t.co/TshyBp120B
Excited to present the work I am cooking up, JACK, today at the "Do Robots Need World Models?", PhysWM, and UncertainWM workshops.
Looking forward to discussing learning from interaction with everyone!
The hottest debate of the year: do robots need π world models π?
Join us at CoRL 2026 for a scientific debate on the case for β and against β world models for robotics, featuring @chelseabfinn@vincesitzmann@YunzhuLiYZ@GeorgiaChal!
π Paper submissions are open now and close October 12. We welcome both research papers and position papers.
π Submit your work for a chance to win an NVIDIA Jetson Thor or a Samsung Galaxy Tab S11 Ultra!
Organized by @longhini_a@wenlong_huang Lasse Peters, @Jefferson_Aero , @PatkiSiddharth@jeff_ichnowski@drfeifei and myself. Thanks to sponsors @NVIDIARobotics and @Samsung!
Do robots need world models? Join us at #CoRL2026 for the workshop on Thursday!
π https://t.co/yoYUliYeBp
π£ November 12 Β· 8:30 AMβ12:30 PM CST
π Brazos room, JW Marriott Austin
Meet Inkling 2 Small, our newest model from
@thinkymachines
β on Tinker today, with weights landing after the completion of our final safety checks π¦Ί
π₯ Diffusion LMs promise what autoregressive models canβt: fixing their mistakes.
But new ones often change the noise and the sampler at once. Which matters?
We tested it on the hypersphere. Noise: 0.8 points. Sampler: 25.
We introduce Spherical Brownian Diffusion π ππ§΅
Arthur is right: Europe has a recurring pattern of defeatism. The key to excellence is momentum: Arthur & Mistral have done a great job getting quite close to the frontier, at very small expenditure (relative to other big labs). Given more runway, they *will* get closer still.
Further, Europes own frontier AI lab is essential for independence. Europe needs frontier labs where young researchers learn the fundamentals of this technology, as a seed for an ecosystem of applications and new startups: a frontier lab is a never-ending source of startups founded by its alumni, value that Europe can only accrue if it has its own frontier lab.
Physical AI - intelligent robots - will require *exactly the same* infrastructure and insights as LLMs do. If Europe gives up on LLMs, Europe will give up its independence on all AI, physical or otherwise.
Lastly, perseverance is a muscle that has to be trained: if Europe doesn't learn to not give up in the face of (fierce) competition in the case of LLMs, it will fail at this exercise again when the next innovation comes around.
Europe needs a frontier lab like Mistral.
yup, curriculum learning is probably as old as RL :D
People usually think of RL as "the agent collects its own data", but the robot can't really pick its own data to solve a hard exploration problem, and see useful gradient signal, if the chance of seeing success is super low (in a sparse reward setting). Which then leads to using denser rewards so that the gradient signal is not impossibly rare to stumble upon. Which then leads to tons of reward tuning and task-specific curriculums. Which then leads to suboptimal policies which are optimizing wrt a soup of mishmashed objectives. Which turns out leads to an inability to scale. So the details of SGS that matter are:
1. Expose a ton of "task configurations" (initial state, goal) to the robot, so that the difficulty of the set of tasks the robot can actually pick from is actually diverse.
2. Instantiate a very simple, automatic, adaptive curriculum based on success rate of each of these task configs (SGS) so that the robot doesn't waste experience in tasks that are neither too easy nor too hard.
So the surprising thing is that this simple instanciation lets us train policies with a sparse reward formulation, at much higher scale than previously possible, to achieve really hard robotic tasks :D
(1/N) Who says you can't do contact-rich, precise manipulation with sim-to-real RL?
No demonstrations, no retargeting, no tactile sensors, no crazy reward tuning. Just 1x, reactive policies trained with scaled up RL.
Excited to finally release Success-Guided Sampling (SGS)! More info in @octi_zhang's thread and more thoughts of my own below! π§΅
Website here: https://t.co/c8vmS1la2M
Do coding agents understand the dynamics of the world well enough to reconstruct it from video?
We introduce 4DCodeBench to evaluate this ability through 4D inverse graphics.
https://t.co/QlkzTPZpbi
<π§΅>
Very excited to share LoGo, my internship project at @theworldlabs on post-training world models! We found that reward design matters a great deal in post-training long-horizon video gen for 3D consistency, echoing what we see in other domains, e.g. LLM reasoning. Check out more π https://t.co/tLW6CVGaAk
Random German lab appears from nowhere and drops SOTA open-weight model
Beats Qwen, Mistral, Nematron models with similar active parameter count
Trained from stratch
The hottest debate of the year: do robots need π world models π?
Join us at CoRL 2026 for a scientific debate on the case for β and against β world models for robotics, featuring @chelseabfinn@vincesitzmann@YunzhuLiYZ@GeorgiaChal!
π Paper submissions are open now and close October 12. We welcome both research papers and position papers.
π Submit your work for a chance to win an NVIDIA Jetson Thor or a Samsung Galaxy Tab S11 Ultra!
Organized by @longhini_a@wenlong_huang Lasse Peters, @Jefferson_Aero , @PatkiSiddharth@jeff_ichnowski@drfeifei and myself. Thanks to sponsors @NVIDIARobotics and @Samsung!
World Labs is joining @AMD!
Over the past few years our incredible team has pushed the boundaries of spatial intelligence through models like Marble, RTFM, and now Atlas.
By partnering with AMD we will train bigger and better models than ever before, and build out an ecosystem of open models. I canβt wait to get started!