numéro9

number 9

[numéro9 @CalvertLewin4]

Already

[numéro9 @CalvertLewin4]

An unfortunate injury in the rhomboid major while enjoying one of the best Saturdays, can't tell how bad it is yet but we wait for monday for the analysis 💔

[numéro9 @CalvertLewin4]

Gutted to announce that After missing the last competition, i won't be abel to have any world cup football.💔

[numéro9 @CalvertLewin4]

@F0reverEvert0n richa & allan

[Andrew Brown @andrewbrown]

If you don't write good docs to power skills and sub agents then this is what you get. When we do the Claude Code Camp all these problems will vanish.

[numéro9 @CalvertLewin4]

@Mugborn5 No

[numéro9 @CalvertLewin4]

@Shruti_0810 Ai

[KJG @MrKJG79]

@Everton

[numéro9 @CalvertLewin4]

@techwith_ram Thanks for sharing

[numéro9 @CalvertLewin4]

@ritu_twts Code blocks

[numéro9 @CalvertLewin4]

@ritu_twts Sublime was peak

[numéro9 @CalvertLewin4]

Inside the LPU: Deconstructing Groq’s Speed https://t.co/hUabbhqwHX

[numéro9 @CalvertLewin4]

@Suryanshti777 ML

[Suryansh Tiwari @Suryanshti777]

🚨 Harvard-level Machine Learning book — FREE

Most people learn ML wrong
They jump into tools
But don’t understand systems

This book fixes that

“Introduction to Machine Learning Systems”
Used by serious engineers

And yes — I’m giving it away for free

To get it:
Follow me (must for DM)
Like + RT
Comment “ML”

I’ll DM everyone who follows

[Bayer 04 Leverkusen @bayer04_en]

The Confederation of African Football (CAF) has declared Morocco – and Eliesse – as winners of the 2025 Africa Cup of Nations! 🏆

[numéro9 @CalvertLewin4]

@tec_marco10 4.6

[Avi Chawla @_avichawla]

Big release from Kimi!

They just released a new way to handle residual connections in Transformers.

In a standard Transformer, every sub-layer (attention or MLP) computes an output and adds it back to the input via a residual connection.

If you consider this across 40+ layers, the hidden state at any layer is just the equal-weighted sum of all previous layer outputs.

Every layer contributes with weight=1, so every layer gets equal importance.

This creates a problem called PreNorm dilution, where as the hidden state accumulates layer after layer, its magnitude grows linearly with depth.

And any new layer's contribution gets progressively buried in the already-massive residual. This means deeper layers are then forced to produce increasingly large outputs just to have any influence, which destabilizes training.

Here's what the Kimi team observed and did:

RNNs compress all prior token information into a single state across time, leading to problems with handling long-range dependencies. And residual connections compress all prior layer information into a single state across depth.

Transformers solved the first problem by replacing recurrence with attention. This was applied along the sequence dimension.

Now they introduced Attention Residuals, which applies a similar idea to depth.

Instead of adding all previous layer outputs with a fixed weight of 1, each layer now uses softmax attention to selectively decide how much weight each previous layer's output should receive.

So each layer gets a single learned query vector, and it attends over all previous layer outputs to compute a weighted combination.

The weights are input-dependent, so different tokens can retrieve different layer representations based on what's actually useful.

This is Full Attention Residuals (shown in the second diagram below).

But here's the practical problem with this idea.

Full AttnRes requires keeping all layer outputs in memory and communicating them across pipeline stages during distributed training.

To solve this, they introduce Block Attention Residuals (shown in the third diagram below).

The idea is to group consecutive layers into roughly 8 blocks.

Within each block, layer outputs are summed via standard residuals. But across blocks, the attention mechanism selectively combines block-level representations.

This drops memory from O(Ld) to O(Nd), where N is the number of blocks.

Layers within the current block can also attend to the partial sum of what's been computed so far inside that block, so local information flow isn't lost.

And the raw token embedding is always available as a separate source, which means any layer in the network can selectively reach back to the original input.

Results from the paper:

- Block AttnRes matches the loss of a baseline LLM trained with 1.25x more compute.

- Inference latency overhead is less than 2%, making it a practical drop-in replacement

- On a 48B parameter Kimi Linear model (3B activated) trained on 1.4T tokens, it improved every benchmark they tested: GPQA-Diamond +7.5, Math +3.6, HumanEval +3.1, MMLU +1.1

The residual connection has mostly been unchanged since ResNet in 2015.

This might be the first modification that's both theoretically motivated and practically deployable at scale with negligible overhead.

More details in the post below by Kimi👇
____
Find me → @_avichawla
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

[Math Files @Math_files]

In order to be born, you needed:

2 parents
4 grandparents
8 great-grandparents
16 second great-grandparents
32 third great-grandparents
64 fourth great-grandparents
128 fifth great-grandparents
256 sixth great-grandparents
512 seventh great-grandparents
1,024 eighth great grandparents
2,048 ninth great-grandparents

For you to be born today from 12 previous generations, you needed a total of 4,094 ancestors over the last 400 years.

Think for a moment:
How many struggles?
How many battles?
How many difficulties?
How much sadness?
How much happiness?
How many love stories?
How many expressions of hope for the future? – did your ancestors have to undergo for you to exist in this present moment...

[numéro9 @CalvertLewin4]

⌚

[numéro9 @CalvertLewin4]

@F1 This has to be tge funniest random pic from last year in Canada 😂😂😂