Great work! An agent that keeps the structure + a video model that renders the pixels is a strong recipe for world models.
In Oneira we made a similar bet: a coding agent maintains an explicit world state, so interactions persist over long horizons.
More: https://t.co/DhvJxnZb2j
Key ideas:
- An extensible world state table for interaction with generated content
- 3D boxes in world coordinates for instance-level interaction
- State updates in the table for long-horizon memory
🌍 Introducing Oneira, an interactive video world model where a coding agent keeps an explicit world state.
Objects that appear as you explore become interactive, and your changes persist over long horizons.
Project: https://t.co/DhvJxnZb2j
Paper: https://t.co/7e4uo2Iqu2
Video world models generate open-ended worlds, but their state stays implicit in latent context. Game engines and simulators make every object's state explicit, though only in a prebuilt world.
Oneira keeps an explicit world state that grows with the generated world.
📢 VLIPP:Towards Physically Plausible Video Generation With Vision and Language Informed Physical Prior 📢
We propose a two-stage approach that integrates physics as condition into VDM to align the generation with physical principles.
🔥Project_Page:https://t.co/xvm7qkiXRt