thank you to the ~800,000 of you that pre-ordered our mobile app on iOS and Android — it’s clear that people are interested in building software on the go
instead of asking you to download yet another app we’ve decided to take an entirely different approach: one where apps emerge naturally, in the course of your everyday conversations with Gemini. we’re partnering with the Gemini app team to make that a reality on mobile and desktop
we’re continuing to invest in the AI Studio web experience for those who want to go from an idea to a prompt to a business
the teams are working hard on bringing this to life, more to share on this later
OpenAI’s internal testing shows that provider-managed conversation state preserves greater continuity across turns and improves performance on long-horizon tasks like ARC-AGI-3. This is a real and useful result. We’re encouraged to see ARC used to identify useful harness design.
ARC’s verified scores use a “no harness” approach to avoid accidental or intentional developer-aware targeting and to fairly compare scores across all providers. All systems receive the same observations, system prompt, and operate under the same action limits. Conversation state is managed client-side using the industry-wide standard interface for LLMs (the OpenAI-style completions API).
We want progress on ARC to reflect true AGI progress, not ARC-specific format training or settings, and we’re actively working with several industry labs, including OpenAI, to figure out how to best incorporate these server-side state management findings into our verified testing setup while remaining fair and consistent across providers.
We’re celebrating the fast adoption of chatGPT Work and all the incredible effort that went into it today. I’m feeling like a limit reset.
Hold on tight to your ultra and /fast and see you in a few hours when I’m back at the laptop!