Alexandre Payot

Optimisation engineer. Plane geek. Certified lurker.

AI Engineer at Graphcore (opinions my own).
@[email protected]

[Graphcore @graphcoreai]

Last week we welcomed the PyTorch London community to our London office for an evening of shared ideas and thoughtful discussion.

Thanks to Arsalan Uddin, Andrew Fitzgibbon and Akshat Tripathi for sharing their insights - and to everyone who brought the conversation to life.

[Graphcore Research @GCResearchTeam]

11 billion parameters. On a phone.

We wanted to see whether Llama-3.2-11B-Vision-Instruct — far too large for a phone in its original form — could fit into a ~4 GB budget.

The answer: yes. Here's how 🧵

[Alexandre Payot @DrPayoto]

@brickroad7 Fab and wafer capacity

[Graphcore Research @GCResearchTeam]

Our latest work uses theory from the '50s to figure out how to design weight quantisation formats for LLM inference.

It's called Optimal Formats for Weight Quantisation and has just hit arXiv.

1/6

[Graphcore Research @GCResearchTeam]

Spring is here and so is Papers of the Month! In this March edition, we cover Transformers without Normalisation, Compute Optimal Scaling of Skills, Overtrained Language Models Are Harder to Fine-Tune, and Multi-Domain Distribution Learning for De Novo Drug Design! 🧵

[Alexandre Payot @DrPayoto]

@EzProgramming @allenholub I think the better way of thinking about it is in the reference frame of the club head: the ball is approaching it at the speed of the club, and bounces off it in the opposite direction.

For a perfectly elastic collision it would go at 2x the club speed (in our reference frame)

[Alexandre Payot @DrPayoto]

@OpenAI Finally! A benchmark that seems to capture the vibe of the last 9 months of "clause sonnet 3.5 is best for coding".

Looking forward to testing other models against it!

[Graphcore Research @GCResearchTeam]

Summary: https://t.co/djqFBtyRr3

Finally, the Phi-4 paper presents a rather different FLOPs angle: spending compute in the data-generation process to create higher quality data, leading to “student” models that (in some domains) out-perform their “teachers”.

[Graphcore Research @GCResearchTeam]

Each month our team writes up summaries and analysis of our favourite ML papers. For December we cover:

The Byte Latent Transformer, Large Concept Models, Memory Layers & Phi-4 — all grouped under the title "Spend Your FLOPs Wisely". Here's what we made of them 🧵

[Alexandre Payot @DrPayoto]

@OndrejMirtes The content is fine but the tone could be nicer at no extra cost, and you might be a tiny bit more likely to get a new contributor 🤞

[Andrej Karpathy @karpathy]

Nice post on software engineering.
"Cognitive load is what matters"
https://t.co/eMgxu0YgWw
Probably the most true, least practiced viewpoint.

[Alexandre Payot @DrPayoto]

@gabriberton Ruff instead of black?

[Graphcore Research @GCResearchTeam]

Our November Papers of the Month is now live.

This edition covers 4 papers, on "super weights", context-parallelism, scaling laws for precision and critical batch sizes. We provide our summaries and analysis of each. (🧵1/n)

https://t.co/CkQySUiNpO

[Charlie Blake @thecharlieblake]

Come join us tomorrow - we'll be talking about the ways in which we've improved muP practically, and made it work (seamlessly) in FP8 🥂

[Josef Dean @JosefNDean]

Sure matplotlib is cool, but what if I want to load my loss curves into the 2006 hit Flash game LineRider?

[AI at Meta @AIatMeta]

We’ve also updated our license to allow developers to use the outputs from Llama models — including 405B — to improve other models for the first time.

We’re excited about how this will enable new advancements in the field through synthetic data generation and model distillation workflows, capabilities that have never been achieved at this scale in open source.

[Clara Dealberto @claradealberto]

Amanda Seyfried as Créations et suppressions nettes d’emplois entre 2008 et 2018 dans les PME (hors microentreprises)

[James Lucas @JamesLucasIT]

Animals interrupting wildlife photographers - a thread 🧵

1. A curious fox by Dan Dinu

[Graphcore Research @GCResearchTeam]

New blog post by @douglasahorr breaking down the components of a modern transformer LLM.

Specifically written for programmers and casual ML enthusiasts, to give an understanding & minimal implementation of what's going on under-the-hood.

https://t.co/COoQiJAYs8