Rohan Doshi

pm lead for gemini multimodal @ deepmind. views are my own.

[Rohan Doshi @RohanLikesAI]

Today, we’re launching Agentic Video Understanding in Gemini across 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite.
📉 ~88% fewer tokens
💰 ~66% lower cost
📈 ~7% higher accuracy
The result: a new Pareto frontier for video understanding across accuracy and cost.

Traditional video understanding processes video statically at a fixed sampling rate. With Agentic Video Understanding, Gemini can actively decide what to watch, at what speed, and which signals to use (frames, audio, or transcripts) by using native video tools.

This enables split-second retrieval, rapid-motion understanding, and needle-in-a-haystack search across hours of video.

Available now via the Gemini API - and coming soon to billions of users through the Gemini App and YouTube’s “Ask YouTube” experience.

Learn more (details, demos, and docs): https://t.co/kOYrewXHYL

PMing this 0→1 has been a career highlight. Taking it from an idea and helping shape it into a frontier-defining capability we’re now shipping broadly has been incredibly rewarding.

Huge thanks to @MarioLucic_ , @skprat @ahmetius , @FPavetic, @suhasyogin , @jalayrac, @bcaine, @nbrichtova, @xu_bibo , @tulseedoshi, @davthack, @koraykv and the multimodal team at @GoogleDeepMind.

[Rohan Doshi @RohanLikesAI]

@jastephx congrats on the launch jason!

[Rohan Doshi @RohanLikesAI]

any feedback y'all have for the multimodal team? what should we push on next after gemini 4 carbon?

[Roboflow @roboflow]

Gemini 4 Carbon is now the top overall vision model on our benchmarks

Text/OCR seems to be a consistent high performing area over others
- text localization and recognition: finding text, drawing boxes, and reading it
- short string extraction: reading a specific value from an image
- long text transcription: reading full passages from images

Beyond that, it's #1 in core vision tasks like object detection and reasoning.

Disclaimer: it is the slowest and most expensive of the top models so if you need cost/speed, this one isn't for you

[Rohan Doshi @RohanLikesAI]

we are just getting started ;)

[SkalskiP @skalskip92]

Gemini 4 Carbon is so fucking good at computer vision

- #1 overall
- #2 object detection
- #1 short string extraction
- #4 structured JSON extraction
- #1 long text transcription
- #1 text localization and recognition

very slow and expensive though...

↓ examples

[Rohan Doshi @RohanLikesAI]

🚀Proud to launch MKA, DeepMind's newest visual knowledge work benchmark, in partnership with Artificial Analysis! Most knowledge work benchmarks can be solved with text. Real professional work can't. MKA tests whether agents can read blueprints, schematics, maps, scientific charts, etc and then produce actual deliverables (a deck, doc, or spreadsheet).

Gemini 4 Carbon is #2 overall, about 2 pts behind Fable 5.2, and costs 6x less on average per task.

I had fun driving this 0→1 with an amazing team 🙏 @bcaine, @weichengkuo, Anima Singh, Ivor Rendulic, Ivan Jurin, Antoine Yang, Oran Kelly, @MarioLucic_, @suhasyogin, @jalayrac, @tulseedoshi, @nbrichtova, @xu_bibo, and many others.

📝 Blog: https://t.co/Rtl80XerYf
📄 Paper: https://t.co/UmRLvnT5GV
📊 AA: https://t.co/BCd6toAqCb

[Artificial Analysis @ArtificialAnlys]

We’ve partnered with Google DeepMind to launch their Multimodal Knowledge Agent (MKA) evaluation, a new benchmark for agentic knowledge work over complex visual files

@GoogleDeepMind’s MKA is a new benchmark requiring agents to review visually complex project files such as schematics and scans, and produce deliverables like slide decks, spreadsheets, reports, and marked-up images

Multimodal Knowledge Agent (MKA) overview:

➤ 200 private real-world visual tasks across 10 industries: Agents work from dense visual project files including architectural drawings, engineering schematics, and clinical scans to produce deliverables including slide decks, spreadsheets, PDFs, Word documents, and marked-up images. Unlike many agentic evaluations, these inputs emphasize realistic visual complexity and domain-specificity, and are genuinely required to complete the task rather than just providing additional context

➤ Climb vs. Regression rubric scoring: Each task is graded against expert rubrics, averaging ~70 weighted criteria per task. Criteria are split into ‘Regression’ (baseline checks models should pass) and ‘Climb’ criteria (harder checks designed to separate strong models). The headline score is Climb WSR (Weighted Satisfaction Rate) - the weighted proportion Climb rubric criteria successfully completed

➤ Evaluated in Stirrup with an agentic judge: We run every task in Stirrup, our open source agent harness, with code execution, web search, web fetch, and image-viewing tools (up to 250 turns), and grade submitted deliverables using GPT-6 Luna (medium) as an agentic judge

Key MKA results:

➤ Claude Fable 5.2 leads on multimodal knowledge work: Anthropic’s recently released Claude Fable 5.2 (max) leads the MKA leaderboard with a Climb WSR of 57%, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%

➤ Cost per Task spans nearly 30× within 10 points of the top score: Claude Fable 5.2 averages $31.97 per task, over 5× Gemini 4 Carbon ($6.29) for ~2 extra points, and ~29× GPT-6.1 Sol ($1.11). Below Carbon, all four OpenAI models tested sit on the cost-performance Pareto frontier (GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6 Luna), with Claude Haiku 5.5 and MiMo-V2.6-Pro making up the rest of the frontier

➤ No single model leads across all 10 industry verticals: Claude Fable 5.2 leads five verticals (Software & Data, Engineering, Logistics & Mapping, Business, and Finance alongside Claude Sonnet 5.5), Claude Opus 5.5 (max) leads Education and Construction, Gemini 4 Argon (high) leads Science, Gemini 4 Carbon (high) leads Healthcare, and Claude Sonnet 5.5 (max) narrowly leads Law & Government

Thank you to the @GoogleDeepMind team for their work building this benchmark and collaborating with us to bring it exclusively to Artificial Analysis!

[Rohan Doshi @RohanLikesAI]

Super excited to finally release access to the Gemini 4 Carbon model for y'all to play with! We also released the Multimodal Knowledge Agent (MKA) benchmark, in partnership with @ArtificialAnlys - sharing more soon!!

[Google @Google]

Gemini 4 Carbon is now available.

It’s our strongest model, built to dig deeper, handle long-horizon tasks, and multi-step reasoning.

If Argon is great for complex tasks, Carbon is the model you bring in for your hardest problems.

[Rohan Doshi @RohanLikesAI]

our multimodal and agentic capability bets are landing! sharing more soon!

[Artificial Analysis @ArtificialAnlys]

Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index

Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning, knowledge, and scientific tasks. It also has large gains in coding and knowledge work, particularly when compared to other Gemini models, but it is not leading in all these areas.

As with launch of Gemini 4 Argon, Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available. Google is offering a 50% launch discount, with no confirmed end date. All our charts and figures use standard pricing for both Gemini 4 Carbon and Argon.

Key results for Gemini 4 Carbon with high reasoning:
➤ Gemini 4 Carbon leads the Intelligence Index v5, with the #1 score in 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA. Carbon makes substantial gains elsewhere but is not leading in coding and knowledge work

➤ Behind only Claude Opus 5.5 on coding agents: With Antigravity CLI, Gemini 4 Carbon (high) scores 59 only 3 points behind Claude Code with Claude Opus 5.5 (max, 62) on the Artificial Analysis Coding Agent Index. Carbon ranks second on both Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%), behind Claude Opus 5.5 (max) at 56%. Google has historically underperformed in this area and so this improvement stands out

➤ Improved agentic knowledge work, but not yet leading: On AA-Briefcase, Gemini 4 Carbon (high) scores 1609 Elo, a 126-point improvement over Gemini 4 Argon (high, 1483), but still behind models including Grok 4.7 (xhigh, 1638), Muse Wonder (max, 1659) and Kimi K3.1 Pro (1678). The largest gain comes from Presentation, up 293 Elo to 1598, alongside a 120-point improvement in Analytical Quality to 1671

➤ Lower Cost per Task than Claude Opus 5.5 for comparable overall intelligence: Gemini 4 Carbon costs $7.76 per Intelligence Index task, ~23% less than Claude Opus 5.5 (max, $10.04) but ~1.2x GPT-6 Astra (max, $6.25). It uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max, 173k). At the same $4/$20 standard pricing, its Cost per Task is in line with Gemini 4 Argon ($7.68), while scoring 9 points higher on the Intelligence Index

➤ Leading AA-Omniscience v1.1 score, with low hallucination: Gemini 4 Carbon scores 54 on our recently updated AA-Omniscience v1.1, 6 points ahead of Claude Opus 5.5 (max, 48) and 8 points ahead of Gemini 4 Argon (46). Accuracy (% correct) rises 10 points over Argon to 58%, while its hallucination rate rises from 4% to 11%. This is still the second lowest of any model, behind only Argon, compared with 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max)

Key model details:
➤ Context Window: 1M tokens
➤ Multimodality: Text, image, video, and speech input, with text output
➤ Pricing: $4/$20 per 1M input/output tokens, the same as Gemini 4 Argon’s standard pricing, with a 50% discount. Cached input tokens receive a 95% discount ($0.20 per 1M)

Congratulations to the @GoogleDeepMind team!

[Ben Caine @bcaine]

Sorry we even made you consider Astra for a few weeks. Won't happen again.

[Rohan Doshi @RohanLikesAI]

excited to share gemini 4 with the public! if you have feedback on the multimodal inputs or outputs, please reach out!

[Logan Kilpatrick @OfficialLoganK]

Today we are rolling out Gemini 4 Argon globally in the Gemini API, to Pro + Ultra in Gemini App, and Ultra subs in Antigravity!

Argon is a huge step forward for Gemini, and a labor of love. This model has been battle tested by over 100,000 people at Google over the last month, with so much new innovation, ambition, and hard work baked into this release. The team has been heads down over the last 6 months working to make this possible across the stack (research, infra, safety, eng, etc).

We rolled the model out to cyber defenders as part of our fairwind program and early access customers over the last 2 weeks, it has been amazing to see and talk to folks who are now using this model daily. Seeing the models writing capability, raw knowledge and persistence for coding, paired with new features like 1M token output when the model needs to work more has gotten me even more excited about our path ahead.

This is just the start of the Gemini 4 era. As you saw with 3.5, 3.6, 3.7, and 3.8 Flash, our iteration speed with Gemini is fast : ) Now with Gemini 4 Argon to work from, I am cannot wait to see the progress continue, and most importantly see what this new model enables for the world.

This is such a special moment in the arc of technology, as millions of people everyday tackle problems they could never have thought possible before. Gemini 4 Argon was built to help you tackle the hardest problems. I have personally felt this spark with Argon and hope you do too. Onwards!!

[Rohan Doshi @RohanLikesAI]

Spent the last few months as the multimodal PM obsessed with one question: can agents work with the files you actually work with? Decks. Sheets. Charts. Blueprints. CAD. Maps. Scans.

Gemini 4 Argon is our first look at the answer. So proud of the multimodal team. Lots more to come.

[Logan Kilpatrick @OfficialLoganK]

Introducing Gemini 4 Argon, our new frontier model, rolling out to cyber defenders starting today, and more widely as soon as possible. I am really excited by the progress we have made here. Argon is priced at $2 in and $10 out during introductory pricing!

[Ankesh Anand @ankesh_anand]

We are back! Gemini4 Argon is an incredible model, super proud of the work we did here!

It’s perhaps the most well rounded model out there: this is reflected on Vals, a gdp weighted suite of evals where it ranks #1. Onwards!

[Rohan Doshi @RohanLikesAI]

@davidpantera_ @GoogleDeepMind @StanfordGSB Welcome to the team!

[Rohan Doshi @RohanLikesAI]

Developer and enterprise adoption of agentic video understanding has been amazing!! Here are some case studies on the new use cases unlocked by this frontier capability (e.g. video editing, video-based app regression testing, etc)

[Google for Developers @googledevs]

https://t.co/nCtQYQr6LZ

[Google AI Studio @GoogleAIStudio]

https://t.co/lAl948negL

[Rohan Doshi @RohanLikesAI]

so many cool use cases for agentic video understanding! ⚽️

[Vamsi Batchu @vamsibatchuk]

this agentic video understanding with Gemini based experiment got a lot of reactions & questions, will share a repo soon, so you can try it out. I have more examples coming in the next few days. there are so many possibilities with this model !!! 👾📹

[Vamsi Batchu @vamsibatchuk]

this agentic video understanding with Gemini based experiment got a lot of reactions & questions, will share a repo soon, so you can try it out. I have more examples coming in the next few days. there are so many possibilities with this model !!! 👾📹

[SkalskiP @skalskip92]

Gemini 3.8 Flash might be my new favorite Gemini model

- #1 image reasoning
- #1 data extraction
- #2 object detection
- 30% faster than Gemini 3.7 Flash

↓ Gemini 3.8 Flash deep dive

[Rohan Doshi @RohanLikesAI]

Excited to share agentic video is also available on 3.8 flash!

[Logan Kilpatrick @OfficialLoganK]

Introducing Gemini 3.8 Flash, another jump in Gemini's agentic + coding capabilities, and our 3rd updated Flash model in only 6 weeks...

This model has been a ton of fun to work with, excited to see what you all think!

[Rohan Doshi @RohanLikesAI]

Super excited to see how Mosaic is leveraging agentic video understanding!

[Adish Jain ☕️ @_adishj]

Mosaic is a Day 1 partner with @GoogleDeepMind and the new Gemini Flash 3.7 Agentic Video Understanding model.

we had the opportunity to test this new model out in early access with our video agents over the past few weeks and the results are stark.

full blog post from @googledevs in the comments! 👇

[Google AI Developers @googleaidevs]

Can Gemini count the number of claps? 👏

Accurately counting rapid movements is a notoriously tricky task for AI. Because static processing ingests video at a fixed 1 FPS by default, split-second movements like a clap easily get missed entirely or get confused with a snap or click.

Watch Gemini 3.7 Flash use the new agentic video understanding capability to accurately identify and count every single clap by automatically adapting the processing speed as needed: