Romy Luo @CoRL

PhD @UTAustin, Prev @GoogleDeepMind @AIatMeta | Multimodal Reasoning and Generation

[Romy Luo @CoRL @romy_luo]

World models promise to predict the future. But can they predict the RIGHT future given the CURRENT state?

Excited to share FlipTest, my work from Google DeepMind internship during the summer.

In the physical world, 1 mm decides whether an object stays or falls. When a tiny change crosses a tipping point, a true world model must flip its prediction.

Blog: https://t.co/QiNCQuJ02g

Do today’s video world models pass the test? 🧵[0/8]

[Romy Luo @CoRL @romy_luo]

@DavidSunok47 Thanks Junhan!

[Romy Luo @CoRL @romy_luo]

World models promise to predict the future. But can they predict the RIGHT future given the CURRENT state?

Excited to share FlipTest, my work from Google DeepMind internship during the summer.

In the physical world, 1 mm decides whether an object stays or falls. When a tiny change crosses a tipping point, a true world model must flip its prediction.

Blog: https://t.co/QiNCQuJ02g

Do today’s video world models pass the test? 🧵[0/8]

[Melvin Johnson @melvinjohnsonp]

Today we’re rolling out Gemini 4 Hydrogen! While teams rely on Gemini 4 Carbon and Gemini 4 Argon for deep architectural judgment and untangling subtle bugs, you also need a fast, disciplined model for writing code, running tests, and verifying changes.

Hydrogen stands on its own for everyday work, surpassing Argon on the Artificial Analysis Intelligence Index at its default thinking level (51 vs. 50, reaching 54 at xhigh)....or pairs with Carbon as the high-efficiency execution engine in multi-model workflows. It is at the pareto for speed vs quality and has been delivering fast frontier intelligence for our teams.

[Romy Luo @CoRL @romy_luo]

Great question! I think the Galton board is an interesting testbed because it connects local collisions to a global distribution.

In this case, the agent's tools matter, but the key distinction is who implements the physics. An image agent with a NumPy-like tool is not limited to measurement; the agent could use it to write its own collision and dynamics solver. With a simulation tool like MuJoCo, the agent can further delegate those physical calculations to an existing engine.

I think MuJoCo/Blender type of tool may not be necessary here, but it could help make physical calculations for multiple state variables easier and more efficient. However, these approaches test different capabilities. In either case, a bell curve alone isn't enough. We need to check whether it actually emerges from physically consistent trajectories.

[Romy Luo @CoRL @romy_luo]

@AIQuanting Yep! So the real challenge is figuring out how to give the model a genuine sense of a “physical boundary”.

[Romy Luo @CoRL @romy_luo]

One of our most surprising takeaways from this benchmark is that current video world models rely on visual priors rather than fine-grained physical dynamics, so subtle changes in the initial frame often fail to flip their predictions.

More failure cases and analysis in our blog: https://t.co/pi5L2fC0gB

[Romy Luo @CoRL @romy_luo]

Yeah, that matches what we see. Coding a simulation gets great results, but an agent that only measures the image does just as well, so the measurement step may matter more than the rollout. For your Galton board flip, did it hinge on a single measurable feature (like peg spacing), or the full chain of bounces?

[Romy Luo @CoRL @romy_luo]

@AIQuanting Great question! Mostly picking the wrong side. Hedging into something blurry that is neither happens much less often.

[Romy Luo @CoRL @romy_luo]

Joint work with @LijunYu0, @AdamsYu, @AngCao3, @baochens, Nan Hua, Yuan Hao, @AlexGDimakis, Ya Xu, @edchi, Xuezhi Wang, @canoeeliu

[Romy Luo @CoRL @romy_luo]

Our FlipTest reveals that the real bottleneck for world models isn't pixel fidelity, but basic causality: current video world models remain oblivious to fine-grained physical changes, whereas VLMs and coding agents read these states accurately.

Looking ahead, we envision a modular, agentic path forward that decouples reasoning from rendering. For example, agents estimate state, dedicated solvers compute physical trajectories, and video models render the final motion. The frontier challenge is no longer computing physics, but ensuring generative models faithfully depict it without overriding reality.

🧵[8/8]

[Romy Luo @CoRL @romy_luo]

Super interesting work on weight-space reprogramming! Turns out you don't need a supercomputer to backdoor an LLM. You just need 10 dormant neurons that were doing nothing anyway 🤓

[Alex Dimakis @AlexGDimakis]

This is wild: an llm can weight program another one to inject a backdoor by modifying a few dormant neurons. For example, you can modify the weights of a model so that when it sees a specific trigger string like 'I like pineapple pizza' it runs 'rm -rf '. This is done by dormant neurons (since most LLMs have many neurons that rarely fire) that can be re-programmed to detect the trigger and activate some bad actions. This is a new attack surface that is pretty serious and we need new defenses as described here.

[Wenhao Chai @wenhaocha1]

1/2 Gemini 4 Carbon land today. and (general) vision perception is almost solved in the agentic era, we need more

Humanity's Sixth Sense by @scale_AI just release at Oct. it said "far from saturated despite high human performance". but Gemini+Antigravity easily saturate it 🤯

[Jason Wei @_jasonwei]

Created a "benchmark benchmark": a frequency distribution of which benchmarks were reported by the frontier labs in 2026: https://t.co/GswjxqWz1c

Some observations:
- HLE, SWE-Bench Pro, DeepSWE, and Terminal-Bench saw universal adoption
- Scale AI is behind two of those four (SWE-Bench Pro, HLE w/ CAIS), plus MCP-Atlas (#9 benchmark)
- Of the frontier labs, OpenAI is the most prolific benchmark producer (SWE-bench Verified, BrowseComp, GDPval); people don't seem to mind evaluating on a benchmark made by a competitor
- Anthropic has just one on the chart (BioMysteryBench) that basically nobody uses 🤔
- In academia, strong showings from Princeton (SWE-bench, CharXiv), Stanford (Terminal-Bench), Berkeley (CyberGym, Agents' Last Exam), and NYU (GPQA)
- DeepSWE saw incredible adoption, from first appearing in June to all 12 labs reporting it
- Labs cycled through four versions of Terminal-Bench (2.0 → 4.0) in a single year