Aleksandar Botev

Research scientist at Google DeepMind.

[Aleksandar Botev @botev_mg]

We present Griffin: A hybrid model mixing a gated linear recurrence with local attention. This combination is extremely effective: it preserves all the efficient benefits of linear RNNs and the expressiveness of transformers. Scaled up to 14B!
https://t.co/uptKapilDM

[Samuel L Smith @SamuelMLSmith]

A big win born in @OpenAI London! 🇬🇧

[Aleksandar Botev @botev_mg]

If anyone is interested on working in an exciting team at the frontier of LLM research in London, please reach out to me or Sam.

[Samuel L Smith @SamuelMLSmith]

The Training team @OpenAI is hiring researchers in London 🚀

Our twin missions are to train better LLMs, and serve them more cheaply

Get in touch if you are excited to collaborate on architecture design, reliable scaling, and faster optimization

[Sophia @sopharicks]

It was a pleasure to host the talk with @botev_mg about the Griffin architecture (an alternative to the Transformer) and recall our internship days at OpenAI. Griffin handles long sequences well and is more efficient during inference. In some use cases, it can replace Transformers. Curious if the industry will adopt the hybrid model (Transformers + alternatives) over the years.

Watch the lecture about Griffin on our YouTube channel:
https://t.co/m95eh8O7GP

#TechTalk #techtalks #MachineLearning #ArtificialInteligence #largelanguagemodels #LLMs

[Sophia @sopharicks]

Excited about the upcoming talks I'm hosting in the next couple of weeks. With @botev_mg, we'll be exploring Griffin, a novel architecture and an alternative to Transformers. And @aahmadian_ from @cohere @CohereForAI will share about a new optimization method for RLHF. The details and registration are in the BuzzRobot newsletter https://t.co/SyQJZuJJGp

[Aleksandar Botev @botev_mg]

Our 9B Griffin model is finally open sourced.

Similar performance to the base Gemma model, but much faster!

Throughput is through the roof 🤯

Available on Kaggle, Huggingface and github!

[Samuel L Smith @SamuelMLSmith]

RecurrentGemma-9B is out!
https://t.co/rSTQn2SlhR
https://t.co/Il5UudfpZk

- Uses Griffin architecture, combining linear recurrence with local attention
- Downstream evals comparable to Mistral and Gemma
- Faster inference, especially for long sequences or large batch sizes

1/n

[Aleksandar Botev @botev_mg]

@JagersbergKnut @burkov I think this depends a lot whether you are looking at latency or throughput, since RG uses a lot less memory, hence can fit larger batch size, which shows up only in throughput.

[Aleksandar Botev @botev_mg]

@JagersbergKnut @burkov Actually both models have roughly 7B non-embedding parameters and around 1.5B embedding parameter, totalling 8.58B each. The only discrepancy, with respect to parameters, is in the naming of the two models.

[Jeethu Rao @jeethu]

Looks like Google has just silently released a 2B recurrent linear attention based model (non-transformer based, aka the Griffin architecture). This is a bigger deal than CodeGemma, IMO. AFAIK, the closest thing to this is RWKV.
https://t.co/u1Wsx47rLQ
https://t.co/p2lQpP1kzC

[Mihir Kale @maninblack815]

Happy to share - blah blah blah.
Gemma + Griffin = RecurrentGemma
Competitive quality with Gemma-2B and much better throughput, especially for long sequences.
Cracked model from cracked team!
Check it out below 👇

[Jyrki Alakuijala 🇺🇦 @jyzg]

Our usually compression-centric team helped with the C++ implementation. Gemma runs on the Highway library originally built for HighwayHash and developed further and opensourced in the JPEG XL effort.

[Nando de Freitas @NandoDF]

I’m very proud of our team for open sourcing RecurrentGemma. Yes, recurrence is back and it results in huge gains at inference time. Just look at the impressive throughput plot below.

For details, please see the paper and GitHub page:

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models by
Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, Caglar Gulcehre

https://t.co/TKswAVFcZ3

https://t.co/I3AVNqp6GF

[Aleksandar Botev @botev_mg]

Huge congrats to the whole team:
@SamuelMLSmith
@AnushanFer61200
@GeorgeMuraru
@haroun_ruba
@LeonardBerrada
@sohamde_
+ many others

[Aleksandar Botev @botev_mg]

Following our previous work, we are releasing RecurrentGemma - a fully open source 2B model based on our Griffin architecutre!

Code + weights as everyone has wished for!

Code on Github: https://t.co/K4Ifc8e4Oe
Weights on Kaggle: https://t.co/wDntCLaZ8Q

[Aleksandar Botev @botev_mg]

We are releasing an optimized JAX implementation with a custom Pallas kernel for TPUs, PyTorch reference implementation and C++ code for the laptop enthusiasts!

Also on Huggingface - https://t.co/chD3wo4kNP and https://t.co/2bL7jZWXFN

[Soham De @sohamde_]

Just got back from vacation, and super excited to finally release Griffin - a new hybrid LLM mixing RNN layers with Local Attention - scaled up to 14B params!

https://t.co/FDyBXyLzAV

My co-authors have already posted about our amazing results, so here's a 🧵on how we got there!

[Lucas Beyer (bl16) @giffmana]

It's not just LLM.

We had essentially final SigLIP models for many months before the paper.

We had essentially final PaLI-3 models for something more than half a year before the paper.

It's not always like this, but if a paper "feels late" it's probably just bigco delays.

[Aleksandar Botev @botev_mg]

@srush_nlp @SamuelMLSmith So Pallas sits on top of Triton and Mosaic, which are the GPU and TPU backends respectively. When we implemented the custom linear scan it doesn't go through Triton at all.

Having said, indeed it just uses the `lax.control_flow.for_loop` primitive which works with references.

[Samuel L Smith @SamuelMLSmith]

@caglarml @sohamde_ @botev_mg @GeorgeMuraru Working with you @caglarml was definitely one of my highlights! I hope we keep collaborating in the future🤞

[Aleksandar Botev @botev_mg]

Thanks to @sohamde_, @SamuelMLSmith
, Anushan Fernando, George Cristian-Muraru,
@_albertgu, Ruba Haroun, Leonard Berrada,
@yutianc, Srivatsan Srinivasan, Guillaume Desjardins,
@ArnaudDoucet1, @davidmbudden, @yeewhye, Razvan Pascanu, @NandoDF, @caglarml!

[Aleksandar Botev @botev_mg]

We present Griffin: A hybrid model mixing a gated linear recurrence with local attention. This combination is extremely effective: it preserves all the efficient benefits of linear RNNs and the expressiveness of transformers. Scaled up to 14B!
https://t.co/uptKapilDM

[Aleksandar Botev @botev_mg]

In order to make all these models efficient we had to undergo significant engineering efforts in both careful model design, considering how we shard our models, as well as a custom Pallas kernel for the RNN scan. This has all been achieved by the work of our whole team.