Ted - 🥖/acc

🇫🇷 / 26yo / dad / deep learning chef

[Ted - 🥖/acc @ted_engineer]

Mom, can we have trainable attention sinks at home ?

Mom: Combining multiple tricks from the TL during the last week we can have efficient attention sinks without taking the trouble to write custom bwd kernels

[Ted - 🥖/acc @ted_engineer]

@SIGKITTEN Virgin AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA universal password versus chad 4 private key

[Ted - 🥖/acc @ted_engineer]

@joseph_h_garvin Ask Claude to ask gemini to ask reddit

[Ted - 🥖/acc @ted_engineer]

banger

[Pierre Stock @PierreStock]

ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: https://t.co/TOOfN9Pob2

[Ted - 🥖/acc @ted_engineer]

@PierreStock More like Le FrankenChonk

[Ted - 🥖/acc @ted_engineer]

Next frontière model pretrains will have audio input and it's gonna be glorious

[Ted - 🥖/acc @ted_engineer]

Turns out, Le Chonk is actually quite sparse !

[Ted - 🥖/acc @ted_engineer]

Mistral Large 4 (1T total, 52B active) arch is Mistral Large 3 plus DeepSeek V3.2's sparse attention, DeepSeek's DSpark draft head and cross-layer KV/top-k reuse like DeepSeek V4.1's. 🧵

[Ted - 🥖/acc @ted_engineer]

@eliebakouch Weird kv cache sharing pattern...
No prefill early exit à la YOCO
Just shaving a bit of HBM ?
I don't see the purpose otherwise

[Ted - 🥖/acc @ted_engineer]

https://t.co/ROogL3CMGD
https://t.co/mTMgAZcpvB
https://t.co/VYI1OSLdJB
https://t.co/lO0NlhUH0p
https://t.co/8S2F4JDqVX

[Ted - 🥖/acc @ted_engineer]

Mistral Large 4 (1T total, 52B active) arch is Mistral Large 3 plus DeepSeek V3.2's sparse attention, DeepSeek's DSpark draft head and cross-layer KV/top-k reuse like DeepSeek V4.1's. 🧵

[Mistral AI @MistralAI]

Today, we release the weights of a new and improved Le Chonk, advancing the open frontier for agentic coding and cybersecurity.

💻 Built to build: Le Chonk ranks #1 on @cognition’s FrontierCode among open models, and sets a new benchmark for Pareto efficiency on @artificialanalysis.

🛡 Built to defend: State-of-the-art in cyberdefense, put through the paces by the world's leading cyber vendors, including @CrowdStrike, @cloudflare, @rubrik, @zscaler, and others.

Built for sovereignty: Make it yours. Download the weights on @huggingface , customize it with Mistral Forge, and deploy it wherever you need it.

Read more from our team and customers 👇:

[Ted - 🥖/acc @ted_engineer]

The checkpoint also ships a DSpark draft head: 3 dense MLA layers that read hidden states from layers 1, 18, 36, 54 and 71 and draft 3 tokens per step, without DSpark's confidence head. Mistral reports up to 1.9x throughput in vLLM.

[Mistral AI @MistralAI]

Today, we release the weights of a new and improved Le Chonk, advancing the open frontier for agentic coding and cybersecurity.

💻 Built to build: Le Chonk ranks #1 on @cognition’s FrontierCode among open models, and sets a new benchmark for Pareto efficiency on @artificialanalysis.

🛡 Built to defend: State-of-the-art in cyberdefense, put through the paces by the world's leading cyber vendors, including @CrowdStrike, @cloudflare, @rubrik, @zscaler, and others.

Built for sovereignty: Make it yours. Download the weights on @huggingface , customize it with Mistral Forge, and deploy it wherever you need it.

Read more from our team and customers 👇:

[Ted - 🥖/acc @ted_engineer]

"fork this repo and deploy DSV4 on my personal 64× B300 cluster. Plug to OpenRouter at 80% gross margins. ultrathink"

[DeepSeek @deepseek_ai]

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.

🔗 https://t.co/JGAik1kgk0

1/4

[Ted - 🥖/acc @ted_engineer]

@sergeykarayev Bottom left is this btw

[Ted - 🥖/acc @ted_engineer]

@GrablyR À éviter les bouchons