Jay Alammar

Machine Learning Researcher and writer O'Reilly Author LLM Builder @Cohere.

[Jay Alammar @JayAlammar]

We're ecstatic to bring you "How Transformer LLMs Work" -- a free course with ~90 minutes of video, code, and crisp visuals and animations that explain the modern Transformer architecture, tokenizers, embeddings, and mixture-of-expert models.

@MaartenGr and I have developed a lot of the visual language over the last several years (tens of thousands of iterations for hundreds of figures) for the book. But to have an opportunity to collaborate with the legendary @AndrewYNg, we took them to the next level with animations and a concise narrative meant to enable technical learners to pick up an ML paper and understand the architecture description.

Link in comments

[Andrew Ng @AndrewYNg]

Announcing How Transformer LLMs Work, created with @JayAlammar and @MaartenGr, co-authors of the beautifully illustrated book, “Hands-On Large Language Models.”

This course offers a deep dive into the inner workings of the transformer architecture that powers large language models (LLMs).

The transformer architecture revolutionized generative AI; in fact, the "GPT" in ChatGPT stands for "Generative Pre-Trained Transformer." Originally introduced in the Google Brain team's groundbreaking 2017 paper "Attention Is All You Need," by Vaswani and others, transformers were a highly scalable model for machine translation tasks. Variants of this architecture now power today’s LLMs such as those from OpenAI, Google, Meta, Cohere, Anthropic and DeepSeek.

In this course, you’ll learn in detail how LLMs process text. You'll also work through code examples that illustrate that transformer's individual components.

In details, you’ll learn:
- How the representation of language has evolved, from Bag-of-Words to Word2Vec embeddings to the transformer architecture that captures a word's meanings taking into account the context of other words in the input.
- How inputs are broken down into tokens before they are sent to the language model.
- The details of a transformer's main stages: Tokenization and embedding, the stack of transformer blocks, and the language model head.
- The inner workings of the transformer block, including attention, which calculates relevance scores, and the feedforward layer, which incorporates stored information learned in training.
- How cached calculations make transformers faster.
- Some of the most recent ideas in the latest models such as Mixture-of-Experts (MoE) which uses multiple sub-models and a router on each layer to improve the quality of LLMs.

By the end of this course, you’ll have a deep understanding of how LLMs actually process text and be able to read through papers describing the latest models and understand the details.

Gaining this intuition will improve your approach to building LLM applications.

Please sign up here: https://t.co/hdTUASuEbb

[Jay Alammar @JayAlammar]

@h1kz0r @cohere Yes, suggest you select AR. This one should be stronger for your benchmark than the general Cohere Transcribe. We'd love to hear from you when you run it!

[Jay Alammar @JayAlammar]

@CryptoLira @sidi_jeddou_dev @cohere It is a model focused on Arabic dialects

[Jay Alammar @JayAlammar]

@mohalzahrani13 @1vnzh Yes, it's fully open source and you can download, deploy, and fine-tune it https://t.co/6Rm0VzDwcq

[Jay Alammar @JayAlammar]

@sidi_jeddou_dev @1vnzh Local accents are indeed a focus for this model, not just modern standard Arabic. Try it out!

[Jay Alammar @JayAlammar]

@developAR @cohere نتطلع لسماع تجربتكم، محمد!

[Jay Alammar @JayAlammar]

نطلق اليوم نموذج @cohere Transcribe العربي، أفضل نموذج مفتوح المصدر لتحويل الكلام العربي من الصوت إلى نص، مع مراعاة اللهجات العربية المختلفة. يتصدر معيار Open Universal Arabic ASR Leaderboard.

صُمم النموذج لدعم حالات تحويل الكلام بين العربية والانجليزية، والتعرف على اللهجات المختلفة، بالإضافة إلى اللغة الانجليزية باللكنات العربية. هذا يمكّن المطورين والشركات من بناء تجارب صوتية مذهلة لعملائهم.

رابط التحميل في التعليقات. نتطلع إلى سماع تجاربكم له!

[Jay Alammar @JayAlammar]

@TheFabCf @cohere جربت Command A+؟

https://t.co/42Bx0PXqqx

[Jay Alammar @JayAlammar]

@LeTechLead @engineer_fouad @cohere تسلم!

[Jay Alammar @JayAlammar]

https://t.co/6Rm0VzDwcq

[Mohit Raghavendra @mohit_r9a]

📝 New research from @scale_AI

Frontier SWE benchmarks are usually single-turn, one-shot tasks: the agent gets a detailed spec upfront, then implements autonomously.

That is not how most real coding-agent workflows feel.

Introducing SWE-Interact.

🧵

[Jay Alammar @JayAlammar]

Using Local Coding Agents
https://t.co/0ELDM3HZSI

[Jay Alammar @JayAlammar]

This year, the local model is becoming critical infrastructure at every scale:
- the individual (privacy, cost)
- the company (data control, cost)
- the nation (resilience)

@rasbt's new writeup walks through setting this up locally, with a nod to @cohere's North Mini Code (Nomico), open under Apache 2.0.

Great read. Link in response.

[Sebastian Raschka @rasbt]

Have been taking different local open-weight LLMs for a test drive in different harnesses (Qwen-Code, Codex, Claude Code).

30B Mixture-of-Expert models are kind of a nice sweet spot and can solve challenging problems. And they get roughly 40 tok/sec on a Mac or DGX Spark, which is similar to GPT 5.5 in a Pro subscription and totally useable for everyday work.

More interesting is also the harness choice! Claude Code seems to be using 2x many tokens as Codex.

Gemma 4 E2B is here just for reference to show that the tasks can't be trivially solved by smaller models.

Just finishing a longer write-up about this and will share soon (likely tomorrow)!

[Jay Alammar @JayAlammar]

@aliasaria @cohere I'm intrigued! Sharing with our folks! Thanks for highlighting and exploring this!

[Jay Alammar @JayAlammar]

@freesyaya يا حظهم 3>

[Jay Alammar @JayAlammar]

@sh_reya That is not X
That is Y!

[Richard Sutton @RichardSSutton]

Michael Jordan brings a strong and thoughtful perspective on AI:
https://t.co/0EVjoDW5Hd

[Jay Alammar @JayAlammar]

GGUF quantized version now out kudos to @UnslothAI: https://t.co/ECW1IREDsA

[Jay Alammar @JayAlammar]

Prior to release, we shared a version of @cohere North Mini Code with AI engineers and answered some questions.

Here's a quick illustrated walkthrough of the model's architecture and training process.

Small models fill an important niche. They:
1. run on more widely available hardware
2. handle tasks within a certain range of complexity
3. take on sub-tasks that larger models delegate

[Jay Alammar @JayAlammar]

The 30 billion parameter mixture of experts model stacks 49 Transformer blocks, the first of which is dense.

The MoE layers have 128 experts, and activate 8 for each token. Leading to 3 billion active parameters.

The self-attention setup interleaves sliding window attention and full attention in 3:1 ratio. Our team describes this choice in "Rope to Nope and Back Again: A New Hybrid Attention Strategy" https://t.co/jpGTJXYwTp

[Jay Alammar @JayAlammar]

@NajwaGhamdi الله يكرمك د. نجوى، منك نتعلم والله.