Gopi

Rare Twitter user

[Tom Yeh @ProfTomYeh]

Transformer by hand ✍️ ~ 6 steps walkthrough below

Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking.

Which of those actually make the car run?

Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model.

So I drew and calculated those two parts entirely by hand.

Goal: push five features through one transformer block, filling in every cell yourself.

1. Given

Five positions of input features, arriving from the previous block.

2. Attention matrix

Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own.

3. Attention weighting

We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on.

4. First layer

Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4.

Note that every position goes through the same weight matrix. That is what "position-wise" means.

5. ReLU

We cross out the negatives. They become zeros.

6. Second layer

Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again.

You have just calculated a transformer block by hand. ✍️

The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run.

💾 Save this post!

#AIbyHand #Transformers #DeepLearning

[Sebastian Raschka @rasbt]

How can an LLM switch between low-, medium-, and high-effort reasoning? And how does an LLM learn to reason more or less?

I put together a “little” article explaining how these effort levels are implemented at inference time and during training.

[AI_Explorer @ai_explorer25]

list of 10 active AI researchers on X:

1.
@karpathy
— Andrej Karpathy
2.
@rasbt
— Sebastian Raschka
3.
@AndrewYNg
— Andrew Ng
4.
@drfeifei
— Fei-Fei Li
5.
@demishassabis
— Demis Hassabis
6.
@GaryMarcus
— Gary Marcus
7.
@ai_explorer25
— AI Explorer
8.
@maximelabonne
— Maxime Labonne
9.
@chipro
— Chip Huyen
10.
@ilyasut
— Ilya Sutskever

[Roan @RohOnChain]

My friend applied to 300 tech jobs in two years. No MIT. No Stanford.

Last month Anthropic offered him $750,000.

I asked him how he broke in from zero.

He sent me the exact video that got him in. Andrej Karpathy's 3-hour course on building a full LLM from scratch.

Anthropic's own researcher shows you how LLMs like ChatGPT & Claude are actually built.

I watched it last night.

Halfway through, I realized it's embarrassingly simple to break into an AI lab.

Bookmark this and read the article below.

• 00:00 - intro to LLMs
• 12:41 - LLM training pipeline
• 31:58 - LLM tools & plugins
• 41:14 - LLM transformer architecture
• 2:19:02 - scaling the LLM

[Kabir Menon @the_psyche_lab]

Crucial reforms needed in Indian Family Law:

1. ​Gender-neutral laws (for DV, child custody, alimony)

2. ​Equal access to protection orders

3. ​Strict penalties for misusing laws

4. ​Legalize prenups

5. ​Streamline no-fault divorces

6. Define wife's marital duties

​Repost.

[Tathvam-asi @tathvamasi6]

Welcome to London which BBC will never show you. But it will write hundreds of articles about how filthy India is.

[Deepika Narayan Bhardwaj @DeepikaBhardwaj]

SAME FEMINISTS who wanted blood of 22 yo Himanshu for a misogynistic joke & made thousands of reels for a week dissing every man of the country

ARE HYPERVENTILATING NOW OVER MEDIA TRIAL OF 20 yo SIYA WHO IS A MURDER ACCUSED & WRITING ESSAYS USING EVERY EXCUSE POSSIBLE TO DEFEND HER

Hypocrisy ki ab koi Hadd nahi hai

[Deepika Narayan Bhardwaj @DeepikaBhardwaj]

Taking a moment to thank this cop who heard a grieving family & decided to investigate the angle of murder in what was reported as an accidental death, without thinking why would a 20 year old girl kill her fiance ?

SP Pune Rural.
IPS Sandeep Singh Gill.

If not for him, Siya Goyal and her lover Chetan would have been planning their perfect murder success party and a holiday

Well done @puneruralpolice 👏👏

#SiyaGoyal #KetanAggarwal

[Deepika Narayan Bhardwaj @DeepikaBhardwaj]

GEMS OF FEMINISM 🤡

[Barkha Trehan 🇮🇳 / बरखा त्रेहन @barkhatrehan16]

Priyanka Deshmukh is celebrating Ketan Agarwal's murder.
@anarkaliofara

No wonder I just said #FEMINISMISCANCER

Join me in reporting her account.

@PuneCityPolice

[rewind @rewind02]

Karpathy says the best engineers right now aren't writing more code

they're engineering the context agents operate in

in 30 minutes, the co-founder of OpenAI explains the discipline he calls "agentic engineering" - coordinating powerful but fallible agents without breaking production quality

- why detailed specs are replacing detailed code
- the verifiability principle that determines what agents can be trusted with
- why he tests agentic engineers by giving them a full project, then trying to break it
- what's still entirely on you: taste, judgment, and the plan agents execute against

step-by-step guide on context and agentic engineering

you'll find it below

[Barkha Trehan 🇮🇳 / बरखा त्रेहन @barkhatrehan16]

Dear FEMINISTS
The silence is deafening. Himanshu Jangra’s career was dismantled overnight for his toxic comments, but Dr. Sejal Pawar faces zero mainstream accountability for publicly joking about violating medical ethics and body donors. Why the selective outrage?

[शून्य @0__Shunya__0]

In India, Punishment depends on who you are, not what you did.

[Gopi @iamGopi12]

@Fssailicense @cfs_telangana @PMOIndia @narendramodi @HMOIndia @AmitShah @MoHFW_INDIA @ArogyaAndhra @Rajitpun @Fssailicense @cfs_telangana @cfs_telangana @PMOIndia @narendramodi @HMOIndia @AmitShah @MoHFW_INDIA @ArogyaAndhra @Rajitpun
@Fssailicense @fssaiindia

[Gopi @iamGopi12]

Hello @paperboatdrinks ,
Is the paper boat Swing coconut water is a coconut water or is it sugar water? You showed only 4.3% coconut water in 1.2 litre bottle.

And it takes completely sugar water, either change name to sugar water or only keep coconut water
@fssaiindia

[Gopi @iamGopi12]

@Fssailicense @cfs_telangana @PMOIndia @narendramodi @HMOIndia @AmitShah @MoHFW_INDIA @ArogyaAndhra @Rajitpun

[Gopi @iamGopi12]

*tastes (not takes)

[Vivo @vivoplt]

best 15 accounts to follow in AI:

@karpathy = LLMs king
@steipete = built openclaw
@gregisenberg = startup ideas king
@rileybrown = vibecode king
@jackfriks = solo apps king
@levelsio = startups king
@marclou = startups king
@EXM7777 = AI ops + systems king
@eptwts = AI money twitter king
@godofprompt = prompt king
@vasuman = AI agents king
@AmirMushich= AI ads king
@0xROAS = AI UGCs king
@egeberkina = AI images king
@MengTo= AI landing pages king

follow them all and learn.

[Tabassum Parveen @Tabbu_ai]

Instead of watching Netflix tonight, watch this 2-hour Stanford lecture.

It will teach you more about how LLMs like ChatGPT and Claude are actually built than most people learn in years working in AI.

Stanford released it for free.

Save this.