Luka Ribar

Research Scientist at Graphcore. Previously PhD & MEng at the University of Cambridge.

[Graphcore Research @GCResearchTeam]

We've written an interactive deep dive on Llama 3.2 Vision, alongside a full plain-PyTorch implementation (link in 🧵)

Here's an attention head from the vision encoder in action - the implicit segmentation is quite impressive!

[Luka Ribar @luka_ribar]

Happy to share our recent work on speeding up long-context LLM generation in llama.cpp! ✨ If you’re interested in inference and implementing efficient attention in C++, check it out here:

https://t.co/bLItHQj7wO

[Luka Ribar @luka_ribar]

You can also read the full paper on arXiv: https://t.co/HzJzK3FZvr

[Luka Ribar @luka_ribar]

Excited to present our SparQ Attention paper tomorrow at @icmlconf !

If you're not around to chat to us in person, check out the recent blog https://t.co/0oLEo7sQXd written by Luke explaining our method for speeding up long-sequence transformer inference!

[Luka Ribar @luka_ribar]

Very excited to talk about the work we've been doing on efficient transformer inference at ICLR & ICML!

[Graphcore Research @GCResearchTeam]

Thrilled to announce that our SparQ paper has been accepted to #ICML2024! ✨🎉

For those who can't wait, we'll also be at the ME-FoMo & PML4LRS workshops next week at @iclr_conf in Vienna. Keen to chat with anyone interested in efficient attention.

https://t.co/82cBVrhBRN

[Graphcore Research @GCResearchTeam]

Our April Papers of the Month is now live 🧐

This month the common thread is efficient LLM inference. Our favourite papers cover speculative-decoding + sparse KV (TriForce), 4-bit quantisation (QuaRot) & dynamic compute allocation (Mixture-of-Depths).

🧵
https://t.co/vnWo2V8sny

[Graphcore Research @GCResearchTeam]

Our latest edition of *Papers of the Month* is now available 📚

These are summaries of our team's favourite papers from March, including a new low-rank training procedure GaLore, and the supposed "Era of 1-bit LLMs" (really 1.58 bits)

Mini-version in 🧵

https://t.co/8NzkK0CGyB

[Graphcore Research @GCResearchTeam]

At 2pm today Graphcore researchers @luka_ribar & @savelichic will be presenting at @letsunifyai's popular reading group.

We'll be covering our recent SparQ paper - a method for increasing LLM inference throughput by sparsifying attention.

Live stream: https://t.co/8XSpilGc08

[Charlie Blake @thecharlieblake]

Proud to have played a small part in this paper - a new method for improving token/sec of transformer inference

The trick: sparse KV-cache access based on current query. Uses two-step top-k to find best KVs, without ever loading the full cache (8x mem reduction)

Go try it out😎

[Ivan Chelombiev @savelichic]

1/n While everybody’s been busy packing for #NeurIPS2023, our team at @graphcoreai has been busy with this beauty. Let me introduce:
✨SparQ Attention✨
TL;DR This is a plug-and-play inference Attention block for pre-trained LLMs, which evaporates the KV cache bandwidth 🧵