Mir Mufaqam Ali

Software Developer, Avid Reader

[Abu Āliyah @ElderImran]

I hope this thread has created more depth into your understanding of the topic.

May Allah make it easy.

End of Thread 🧵

[subho ghosh @SubhoGhosh02]

Good stuff I came across today!

[Marius Vach @rasmus1610]

That was one of my favorite papers from last year

[Huaizheng Zhang @zhzHNN]

I always love nano projects as big projects add lots of non-core code to cover every scenario — it often distracts beginners from the fundamentals.

Here are some LLM infra nano projects I recommend you try.

[Dami’ Adenuga @DAMIADENUGA]

This is so creative! Never seen anyone teach like this before

[Vincent Weisser @vincentweisser]

holy duo

[Vivek Galatage @vivekgalatage]

Since the post resonated, check the dedicated chapters on CPU Caches from the book, Algorithmica by Sergey Slotin.

https://t.co/tGEK8bCu4w

[Jerason Banes / Architect of Convirgance @ClassicGamerTWR]

It’s a bit easier to just explain how we got here.

In the old days, memory and CPU ran at the same speed. The laws of physics make it incredibly hard to ramp up the MHz of the CPU while keeping physically separated memory chips in sync. The distance is just too far for the electricity to travel in time available.

The solution was to manufacture a small amount of memory on the chip that is in sync with the chip. Then access the rest of the memory as an external device. Sort of like a linear hard drive. That on-chip memory became the L1 cache and the external chips became DRAM main memory as we know it today.

The second problem was the sheer size of RAM was making it hard to keep the CPU fed. The amount of RAM in the L1 cache was too small. And making it bigger wasn’t a very good option from a price and scaling perspective.

So a faster bit of RAM that was physically closer to the CPU was invented. Slower than L1, but faster than main memory. This sped up operations that were too large to fit in the L1.

Thus L2 was born and eventually moved onto the CPU die rather than being a separate chip on the motherboard.

The final piece of the puzzle isn’t about individual CPU performance at all. As CPUs became multi-core, the cores would fight over which one was accessing main memory. A bit of a “dining philosopher’s” problem.

Even worse, if data was moved between cores, the first core had to send it all the way back to main memory before the second core could read it.

The solution was to create a large chunk of fast memory that was shared between the cores. This acted as a staging point to share memory, and also acted as a place to buffer copies from main memory. The staging point could request a bit more than needed from main memory under the theory that reads would stream from main memory. That way data was available before the core even needed it.

This space became the L3 cache. The only singular cache in today’s CPUs. Each core has their own L1 and L2 caches, meaning there are as many copies of L1 and L2 as there are cores.

And that’s how we got to today’s 3-level cache.

[Kevin Patrick Murphy @sirbayes]

I agree with @karpathy 's take here. The interview between @RichardSSutton and @dwarkesh_sp was interesting, but I think at times there was a communication gap due to some misunderstandings.

I would say that the current LLM training setup is very similar to the classic model-free RL setup, except that with LLMs:
(1) the policy is warm-started from a supervised model (no de-novo, self-directed learning);
(2) there is a train/test distinction (no continual learning);
(3) most of the observation stream comes from human words, which already "carve nature at its joints", bypassing the harder problem of learning useful abstractions from raw sensorimotor streams.
(4) when using multimodal models, the perceptual encoder is usually pre-trained and frozen, and often relies on a lot of human engineering (eg contrastive losses, or pixel-prediction losses) to come up with a good set of (soft) tokens.

Most of the interview seem to focus on issue #1. However, the discussion seemed confused here due to the fact that LLMs are both a world model (predict what humans would typically say) and a policy (predict what the agent should do).

Obviously the model from the supervised pretraining stage is not action-conditioned, so Sutton does not want to call it a WM - but it is a predictor of future observations given the past, so it's like a WM that marginalizes over actions (resulting in a mixture).

The WM is then converted into a (goal-conditioned) policy using IFT (imitation learning) and then improved with RLFT, which further confuses the discussion. In current practice, the RLFT stage mostly just uses human provided reasoning tasks, which are bandit problems that do not involve interacting with an environment. But there is a recent move towards true multi-step RL, where LLMs do learn from external environments, as in classic RL. This fact was not emphasized enough in the interview, IMHO.

Andrej argues that warm-starting is a practical alternative to evolution's outer meta-learning loop, and I agree, so I don't have a problem with #1. But I do agree with Sutton's criticisms #2-#4.

In particular, I expect a lot of future progress to come from continual RL applied to multimodal problems (eg. visual GUI-using agents) in non-stationary multi-agent environments (e.g., e-commerce or embodied AI), where the agent learns its own abstractions over time (eg creating tool libraries), it learns both a (goal agnostic) world model and a (goal conditioned) policy (so it can do decision time planning), and both kinds of model become semi-parametric (eg. combining memories and ICL with gradient-based weight updates).

Future agents will not just be a frozen "omni-transformer", consuming and generating tokens, they will be heterogeneous adaptive systems, with many different specialized modules, more like the brain. (This may make serving hard, but who said intelligence would be easy to reproduce?) I think Sutton will like this new paradigm more :)

[Elliot Arledge @elliotarledge]

everything you need to get started in one repo

[You Jiacheng @YouJiacheng]

BTW, They released a deep dive on FP8 KVCache of main MLA.
https://t.co/1jse6UM6rS
so, actually ≈1/5 compared to FP8 dense MLA.

[nathan chen @nathancgy4]

sharing a paper i learned lots from. it just won neurips oral and will be an extremely good read to gain intuitions about:

- attention gates & sinks
- non-linearity, sparsity, & expressiveness in attention
- training stability & long-context scaling

some takeaways :)
what actually changed and why it matters
> the simple tweak is a head-specific sigmoid gate applied after sdpa (G1 in the figure below); this inserts a non-linearity between W_v and W_o, which turns a low-rank linear stack into something more expressive without heavy compute.
> for the gate: multiplicative performs better than additive, and head-specific performs better than head-shared.
> this gate brings consistent wins across moe and dense settings: lower ppl and modest but broad benchmark gains, while also nearly eliminating loss spikes during pretraining (i didn't expect this while reading).
> the stabilizing effect lets you push lr and batch size further than the baseline without divergence (e.g. double the lr rate); in practice the wall-time overhead is reported as <2%, which is a good trade.

the key intuition: sparsity and query-dependence
> the most effective variant produces sparse gate scores concentrated near zero, and crucially they are computed from the current query, so they suppress context that is irrelevant to that token.
> head-specific sparsity matters: sharing gates across heads raises average scores and blunts gains, consistent with heads specializing in different roles.
> this query-dependent sparsity reduces the “attention sink”: avg attention to the first token drops from ~46.7% to ~4.8% across layers (see figure below), and massive activations shrink alongside it.
> interesting negative result: gating only on values (key/value-dependent) reduces activations but does not fix the sink, which suggests that massive activations are not a prerequisite of attention sinks.

downstream behavior
> long-context extrapolation improves: after extending rope and applying yarn to 128k, the gated models retain much more of their within-32k quality and pull ahead at 64k/128k.
> my read is that input-dependent gates adapt more gracefully than relying on softmax’s normalization quirks.
> the gate seems to act like a dynamic “do-nothing” option per head, which reduces pathological activations and makes bf16 training tamer.
> would assume that tiny parameter overhead and negligible latency make this an appealing default when building long-context or high-lr schedules.

[Max Mynter @MaxMynter]

You asked, I deliver (2 weeks late).

A blogpost about what I did over the last 18 months to sign as a research engineer at a foundation model lab.

Link and summary below

1/n

[Mir Mufaqam Ali @iammirali]

@systemdesignone Playbook

[Kevin Patrick Murphy @sirbayes]

Finally, a good modern book on causality for ML: https://t.co/jsHdpOEb96 by @eliasbareinboim. This looks like a worthy successor to the ground breaking book by @yudapearl which I read in grad school. (h/t @JoshuaSafyan for the ref).

[pdawg @prathamgrv]

r/openai
r/aiagents
r/researchML
r/neuralnetworks
r/languagetechnology
r/learnmachinelearning

thank me later

[Idris @7signxx]

Try this powerful Du'a during hardship and distress. Wallahi it works wonders...

[Sarki. @Waspapping_]

Dembele’s wife is honestly the first wife of a Muslim footballer I’ve seen who truly looks and dresses like a Muslim. You look at the wives of other Muslim players like Salah or Benzema &amp; you start wondering if they’re even Muslims

All agendas against Dembele are hereby dropped

[Inside the Haramain @insharifain]

O Allah, You know the calamity and harm that has befallen our people in Palestine, and You alone are capable of removing and relieving it. Nothing is beyond Your power in the heavens or the earth.

O Allah, lift their suffering, break their siege, and be for them a Helper, a Supporter, a Sustainer, and a Protector.

O Allah, they are hungry—so feed them; they are thirsty—so give them drink; they are barefoot—so carry them; they are unclothed—so clothe them; grant them safety from fear, strengthen their hearts, and grant them victory over those who wronged them.

O Allah, O Supporter of the oppressed, O Strong, O Compeller, grant them victory over their and Your enemy—the oppressive Zionists.

O Allah, scatter their ranks, divide their unity, and seize them with the grip of the Almighty. Show us in them Your power that is never turned back from the wrongdoing people.

O Allah, do not raise for them a banner, do not let them achieve their goals, and make them a lesson and a sign.

O Allah, hasten Your relief, and decree for our people in Palestine a victory that raises their banner, establishes their rights, restores their land, security, and dignity—with Your mercy, O Most Merciful of the merciful.

[🥱 Sleepy (ML/DL) @KrishnaNaraKun]

"Complete Maths Roadmap for Machine Learning "

There are 3 pillars for maths involved in Machine Learning -
1) Algebra
2) Probability Theory
3) Calculus