Basil Mustafa

ML research @ google deepmind Zürich
vision things in gemini
opinions are mine but registered copyright of my mother

[Basil Mustafa @_basilM]

🔥

[Arena.ai @arena]

BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆

Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer Query, and Multi-Turn!

Massive congrats to @GoogleDeepMind for this incredible Arena milestone! 🙌

More highlights in thread👇

[Kevin Patrick Murphy @sirbayes]

Gemma 3 is best in class for a VLM that runs on 1 GPU. Should make RL fine tuning feasible. Also Academic researchers can apply for Google Cloud credits (worth $10,000 per award) to accelerate their Gemma 3-based research.

[Basil Mustafa @_basilM]

native image generation in gemini!
this has been a tireless effort from AWESOME people - the list is too long, I am not on it, just hyped to be working with such incredible people.

Have fun with it, but hold your breath for more - this is just the beginning

[Robert Riachi @robertriachi]

some cool examples with Gemini 2.0 native image output 🧵

[Fabian Mentzer @mentzer_f]

Chilling more

[Basil Mustafa @_basilM]

Awesome work as always from the legendary @ibomohsin

[Ibrahim Alabdulmohsin | إبراهيم العبدالمحسن @ibomohsin]

🔥Excited to introduce RINS - a technique that boosts model performance by recursively applying early layers during inference without increasing model size or training compute flops! Not only does it significantly improve LMs, but also multimodal systems like SigLIP.
(1/N)

[Basil Mustafa @_basilM]

amazing work from video understanding jesus @AntoineYang2 alongside @MarioLucic_ @FPavetic @skprat and many others! they've been bringing better, faster video reasoning to a whole new level and have so much more in store ✨🚀♊

[Antoine Yang @AntoineYang2]

Gemini 2.0 Flash's video understanding is here 🚀
Think: search in videos via timecodes, extract text from moving camera footage, analyze screen recordings in real-time interactions with native audio out 🔊
Come and try it https://t.co/Z9zVQbNBUD 😀
https://t.co/Axa4IVplCo

[Ethan Mollick @emollick]

Dang, Google's veo 2, same prompt.

[Michael Tschannen @mtschannen]

Have you ever wondered how to train an autoregressive generative transformer on text and raw pixels, without a pretrained visual tokenizer (e.g. VQ-VAE)?

We have been pondering this during summer and developed a new model: JetFormer 🌊🤖

https://t.co/ngvPzZvUYW

A thread 👇

1/

[Basil Mustafa @_basilM]

I cannot recommend this enough. Paul is absolutely legendary, the whole team in London is fantastic and the vibes are impeccable. You will learn so much and accomplish awesome things. Apply!!! 🚀♊

[Paul Michel @pmichelX]

Interested in working on Gemini pre-training?

I'm hiring a research scientist to work on pre-training data @GoogleDeepMind in London: https://t.co/TZZFUZ1CmP

I am unfortunately not at #NeurIPS2024 but feel free to reach out to ask questions or see the team at the booth there!

[Oriol Vinyals @OriolVinyalsML]

Gemini 2.0 Flash ⚡️ has arrived!

2.0 Flash > 1.5 Pro (again!) 📈
Interacts with a browser 🤖
Native image generation 🖼️
and much more!

Try it out https://t.co/IMN3bqKgJS

As a preview of what is possible, wishing you all a Drastic Holiday powered by 2.0!

[Basil Mustafa @_basilM]

Gemini's native audio capabilities are often overlooked - but very cool imo!

[Alexander Chen @alexanderchen]

Having fun playing with new native audio capabilities in Gemini 1.5 Pro! ♊ Here’s a demo using audio from the #GoogleIO keynote with examples you can try: transcription, word-level timecodes, and searching audio by drawing. (🔊Video has sound)

[Basil Mustafa @_basilM]

@giffmana @HamelHusain @_arohan_ @OfficialLoganK ah I see - the obvious solution is to rename it gemini 1.44

[Basil Mustafa @_basilM]

@giffmana @HamelHusain @_arohan_ @OfficialLoganK frankly lucas I am shocked you are wasting Google's precious money by offering 501.5 when 501.44 would've done the job

[Basil Mustafa @_basilM]

🔥💪🏾♊😤

[Jeff Dean @JeffDean]

Gemini 1.5 Model Family: Technical Report updates now published

In the report we present the latest models of the Gemini family – Gemini 1.5 Pro and Gemini 1.5 Flash, two highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.

Our latest report details notable improvements in Gemini 1.5 Pro within the last four months.

Our May release demonstrates significant improvement in math, coding, and multimodal benchmarks compared to our initial release in February.

Furthermore, the 1.5 Pro Model is now stronger than 1.0 Ultra.

The latest Gemini 1.5 Pro is now our most capable model for text and vision understanding tasks, surpassing 1.0 Ultra on 16 of 19 text benchmarks and 18 of 21 of the vision understanding benchmarks. The table below highlights the improvement in average benchmark performance for different categories in 1.5 Pro since Feb, and also shows the strength of the model relative to the 1.0 Pro and 1.0 Ultra models. The 1.5 Flash model also compares very well against the 1.0 Pro and 1.0 Ultra models.

One clear example of this can be seen on MMLU

On MMLU we find that 1.5 Pro surpasses 1.0 Ultra in the regular 5-shot setting scoring 85.9% versus 83.7%. However with additional inference compute, via majority voting on top of multiple language model samples, we can get a performance of 91.7% versus Ultra’s 90.0%, which extends the known performance ceiling of this task.

@OriolVinyalsML and I are very proud of the whole Gemini team, and it’s fantastic to see this progress and to share these highlights from our Gemini Model Family.

Read the updated report here: https://t.co/CTzTHND4nQ

[Pietro Schirano @skirano]

I was so impressed with the Astra demo at Google I/O yesterday that I decided to build my own version using Gemini 1.5 Pro Flash.

It's so fast and really good. ⚡️

It was even able to detect the gate! Content is streamed directly from my camera.

Voice via @elevenlabs

[Jacob Austin @jacobaustin132]

This is something I've worked on for a while! You can save the activations of one LLM call and reuse them for a follow-up that overlaps with the first.

This means asking a question about a big codebase can take 30 seconds the first time and 1s after that!

[Basil Mustafa @_basilM]

@b00ml00p @mmmbchang from the inside of gemini, this is the demo we all wanted too!!!

[Basil Mustafa @_basilM]

legendary demo from @mmmbchang , no editing, no frills - the interactive agent magic actually at work, fruits of Michael's work alongside an awesome team of geniuses. 🔥♊💪🏾

[Google DeepMind @GoogleDeepMind]

We watched #GoogleIO with Project Astra. 👀

[Basil Mustafa @_basilM]

Y'all this is literally 100% unscripted live content.

Michael shared this with us while the GPT4o announcement was underway. It's the real deal!

[Michael Chang @mmmbchang]

Gemini and I also got a chance to watch the @OpenAI live announcement of gpt4o, using Project Astra! Congrats to the OpenAI team, super impressive work!

[Basil Mustafa @_basilM]

lucas a.k.a. open source santa coming through again!

[Lucas Beyer (bl16) @giffmana]

We release PaliGemma. I'll keep it short, still on vacation:

- sota open base VLM designed to transfer quickly, easily, and strongly to a wide range of tasks
- Also does detection and segmentation
- We provide lots of examples
- Meaty tech report later!

https://t.co/TClPDqKLrK