@LuLing26466911@zhou_xian_ He confirmed it in this tweet
https://t.co/CAOzr9QZCa
"and a upper level LLM/VLM agent that uses the physics engine as tools to compose worlds and generate other modalities of data."
I'm working on exactly this so I was able to guess correctly.
Hi Jon, thank you for your interest! The project is essentially two parts: a low level physics engine that integrates a wide range of solvers for simulating materials with different physical properties, and a upper level LLM/VLM agent that uses the physics engine as tools to compose worlds and generate other modalities of data. The generative agent incorporates multiple generative modules handling different modalities in a modular way, and the high-level goal is instead of generating everything end-to-end, we generate them in a structured way and all of them collectively form a physical world :) Let me know if you want to dive deeper
Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks.
It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task in compute ) and 87.5% in high-compute mode (thousands of $ per task). It's very expensive, but it's not just brute -- these capabilities are new territory and they demand serious scientific attention.
@LuLing26466911@zhou_xian_ Probably not. I donโt think they use diffusion-based priors at all. Itโs likely an LLM based generation with visual eval.