Mohamed Slaoui

[Awni Hannun @awnihannun]

The sparse attention in the new DeepSeek v3.2 is quite simple. Here's a little sketch.

- You have a full attention layer (or MLA as in DSV3).
- You also have a lite-attention layer which only computes query-key scores.
- From the lite layer you get the top-k indices for the each query
- You use those top-k indices to limit which keys and values you use in the full attention layer.

The lite-layer has to be quite fast because it's still quadratic. In the case of v3.2 it has only one key head and a head dimension of 128.

Would love to see a more detailed comparison of state-size / computational efficiency to sliding window attention with sinks and hybrid layers like Mamba2.

[Thinking Machines @thinkymachines]

LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.

https://t.co/fYV4FPi71m

[Richard Sanders @PulaRJS]

Israel murdered a Reuters journalist. In a hospital.

As rescue workers tried to recover his body, Israel murdered them. Live on TV. Along with three more journalists.

Apparently "it's complicated".

[Paul Graham @paulg]

It's so painful to watch these scenes, but we can't just turn away.

[Google DeepMind @GoogleDeepMind]

What if you could not only watch a generated video, but explore it too? 🌐

Genie 3 is our groundbreaking world model that creates interactive, playable environments from a single text prompt.

From photorealistic landscapes to fantasy realms, the possibilities are endless. 🧵

[Mohamed Slaoui @MoSlaoui]

@kimmonismus Yes

[Mohamed Slaoui @MoSlaoui]

@marouane53 Totally fair—but let’s assume every chat has around 500 tokens. With 1B chats/day and 40% being polite, that’s 200B tokens/day. At $2.5 per million tokens, that’s $500K/day. If the model’s pulling that full context, over 900 days you’re looking at $450M

[Bernie Sanders @BernieSanders]

Trump is trying to deport a Columbia Univ. student who has been a permanent resident in the U.S. since she was 7.

Her "crime”?

Attending a protest against the war in Gaza.

No, Mr. President. This is a democracy. You can't exile political dissidents. Not in the United States.

[Richard Sutton @RichardSSutton]

Modern Americans, you are not responsible for slavery. You are not responsible for stealing the land and killing almost all native Americans. Those things happened long ago, before you were born. But you are responsible for the stealing of land and the genocide of Palestinians by the state of Israel. These are directly caused by your bombs and your votes today.

[Richard Sutton @RichardSSutton]

Free Palestine.

[missfalasteenia @missfalsteenia]

Palestinian children celebrating the ceasefire. My heart is so full. 🥹

[B.M. @ireallyhateyou]

This is what we need to talk about. The holocaust in Gaza. Not some imaginary pogroms.

[jason @jxnlco]

anyone:
every experienced ai engineer:

[أنس الشريف Anas Al-Sharif @AnasAlSharif0]

Israel is burning Palestinian civilians alive in Gaza

[Globe Eye News @GlobeEyeNews]

Hollywood Actor Andrew Garfield:

We should be putting our energy towards something that actually matters. Maybe the lives of Palestinians and Gaza right now, and anyone suffering under the weight of the horrors of our world right now.

[Nour Naim| نُور @NourNaim88]

🚨 HEARTWRENCHING: A will found beneath the rubble of Rasha, a child only 10 years old, who was killed by the Israeli army in #Gaza:

“Please don’t cry for me, as it would make me sad. I hope my clothes can go to those in need, and my accessories to Rahaf, Lana, and Batool. The bead boxes should go to Batool. As for my monthly allowance of 50 shekels, I want half to go to Rahaf and the other half to Ahmad. I’d like Batool to have my toys. Lastly, please don’t shout at my brother Ahmad. Please follow these wishes.”

[Bassem Youssef @Byoussef]

Kill. Steal. Expand. Settle. Push people to the brink. They resist, they fight back. I need to defend myself. Kill more . Steal more. Expand more settle hard. Rinse and Repeat. Israel in a nutshell.

[Jake Shields @jakeshieldsajj]

Gad Saad is allowed to make death threats on X and stays monetized

Why do so many of them act like this?