💬 AI assistants need to reason while the conversation keeps changing.
Our new survey maps open problems and new research directions for Native Real-Time Interactive Omni Intelligence.
🌐https://t.co/sXhy9w7mbr
🗂️ https://t.co/1EQvvtMmhi
📙https://t.co/dn3fqeTFbf
PRs welcome!
Amazed by Astra controlling robots? It turns out other LLMs can do it too – even the text-only ones like DeepSeek. We also find Astra did best watching and correcting every move live; Fable did best building its own tools and policy in code first.
We introduce SEAR, a new benchmark and framework to rigorously evaluate and study how LLM agents self evolve on robots.
🔁📈The loop: within a controlled time budget, an agent and its library update a policy → the policy runs in simulation or on hardware → a judge returns feedback → the library carries the skills and lessons into the next task.
💻🚏 Self-evolving agents operate in myriad ways - we study all of them at once: the policy can be code the agent wrote (code-as-policy), the agent itself in the control loop (agent-as-policy), or a VLA it trained (autoresearch).
🧑🔬🤖We spent 4+ years of accumulated agent evolving time across simulation and real hardware and tested 7 frontier models (GPT-6-Astra, Fable 5.1, Opus 5, GPT-5.6-sol, Gemini-3.6-flash, Kimi-k3, DeepSeek-v4-flash) on 330 sim tasks, 2 physics engines, and real robots with different embodiments.
🌐Project overview: https://t.co/IbEoJZUAqA
A thread 🧵1/5
Okay they really use pass@10086! 300B token could even be used for pretraining lmao.
“Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens. In the process of resolving the Navier–Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens.” https://t.co/9JvL5AM3Fm
ByteDance has just released Seed 2.1, which achieved outstanding performance on two of our benchmarks: FrontierCS (https://t.co/1Oo1J0GttV) for frontier computer science problems, and WorldBench (https://t.co/BRNM4c5tsR) for multimodal knowledge.
Model Card: https://t.co/0H0wtCTw97
Manually annotating this benchmark was definitely a painful process. We made sure that:
1. the questions can stump the vast majority of VLMs;
2. we do not ask overly tricky questions that do not exist in real-world scenarios;
3. the questions are diverse.
During annotation, we found that the scenarios where VLMs fail most easily are still perception-related problems, such as counting, while they perform better on questions involving world knowledge. We also found that many existing benchmarks commonly suffer from substantial annotation errors and homogeneous images.
enjoy WorldBench!
Sometimes I feel a bit anxious. About 90% of my coding is now handled by Codex and Claude Code, and it feels like the thing improving rapidly isn’t my own skill level, but the AI’s.
I've become a pretty heavy Codex user lately.
It started as a coding helper, but now it’s part of my daily workflow of automating slack, notion, email, calendar, github and computer/browser-use tasks.
Already saving me at least 30+ min/day. And now I can use it from my phone:)
It's time to systematically teach VLMs to see with synthetic images!
We built VisionFoundry, a simple but intuitive framework that generates synthetic image datasets from only a task name.
10k synthetic data → over +10% improvement on visual perception benchmarks 👀