SoftMax

In God We trust all others must bring data and memes

[merve @mervenoyann]

if you want to get every last bit of performance out of your local setup, you need to know a bit about inference 🥵

but we've got you covered, shipping conceptual guides 🔥

here's the first one about prefill, decode, KV cache what you should optimize for 🙌🏻

[Saurabh Kumar @drummatick]

one of the best videos on FlashAttention

[Andrew Ng @AndrewYNg]

New: A map of the most important skills in AI Engineering. https://t.co/VVkn1Dqp1N

[merve @mervenoyann]

I shipped a fine-tuning tutorial for Muse Glimmer 30B on AI2's MolmoWeb dataset with TRL

https://t.co/VstYoch3le

> but merve, this model is sota on ScreenSpot-Pro?
yes, but it doesn't work well with ambiguous prompts of MolmoWeb that are close to how you interact with computer, e.g. "jump to nutrition facts"

I compared model fine-tuned on MolmoWeb format against base model zero-shot outputs converted to MolmoWeb format (coordinates on 100)
I found that base model has 13% click accuracy + 35.0% within 5% diagonal while fine-tuned model has 41% + 68% within 5% diagonal

[leonie @helloiamleonie]

The @liquidai cookbook is such an underrated developer resource:

Curious about fine-tuning text, vision, audio, or encoder models?
Curious about fine-tuning with CPT, SFT, DPO, or GRPO?
Curious about fine-tuning LFMs with Unsloth or TRL?

It has it all.

I just did a little cleanup. Enjoy!
https://t.co/y4mJxrQwDC

[Lilian Weng @lilianweng]

A super long overdue (3+ years?) post on scaling laws.

Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.

The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.

https://t.co/HP26eJvjHB

[Liquid AI @liquidai]

Introducing LFM2.5-Embedding-350M and LFM2.5-ColBERT-350M: two multilingual retrieval models built for ultra-fast and accurate search across 11 languages.

> End-to-end retrieval latency as low as 1.5ms with our enterprise stack! 🚀

> Consistently best-in-class multilingual and cross-lingual performance across Arabic, German, English, Spanish, French, Italian, Japanese, Korean, Norwegian, Portuguese, and Swedish.

🧵

[Vlad Feinberg @FeinbergVlad]

How to land a job at a frontier lab

https://t.co/oHIqLgBMbC

[Daily Dose of Data Science @DailyDoseOfDS_]

NVIDIA & Unsloth dropped one of the best practical guides on building RL environments from scratch, filling gaps most tutorials skip.

Covers:

- Why RL environments matter + how to build them
- When RL beats SFT
- GRPO & RL best practices
- How verifiable rewards & RLVR work

[Victor M @victormustar]

Very hyped by the new Cohere Transcribe model 🌍

Works surprisingly well on bad quality audio when the mic doesn't cooperate. 2B params, 14 supported languages and it's Apache 2.0.

try the official Hugging Face demo ⬇️

[Joe @joedab12]

If you guys aren't following the CEO of Cerebras-you should be. One of the brightest guys in the industry. Great writeup for the technical analysis guys who know nothing about AI. I'm somewhere in the middle and learned a lot here.

@andrewdfeldman would love to hear your thoughts on power density and last inch power delivery. Why did Cerebras decide to use Vicors' vertical power delivery?

$NVDA $VICR $MU $AMD $RMBS $BE $AAOI $SNDK

[Avi Chawla @_avichawla]

https://t.co/zu8HupqrL5

[ali @waterloo_intern]

- 230 training runs
- 1,623 GPU hours (67 B200 days)
- 76 TB of training data
- a 2x faster model

Every paper said it can't be done.
Quantization Aware Distillation made it possible.

[AVB @neural_avb]

My timeline feels so blessed today. Full of deep learning maniacs writing cool articles like this. 👀

[vLLM @vllm_project]

Maintaining separate attention kernels for every GPU platform doesn't scale.

The vLLM Triton attention backend takes a different approach: ~800 lines of Triton, same source code across NVIDIA, AMD, and Intel GPUs. On H100, it matches state-of-the-art attention performance. On MI300, ~5.8x faster than earlier implementations.

The blog covers Q blocks for better dot utilization, parallel tiled softmax for decode, persistent kernels for CUDA graph compatibility, and cross-platform benchmarking.

Now the default attention backend on AMD ROCm, also available on NVIDIA and Intel.

📖 Deep dive: https://t.co/De58RQuvu5

Thanks to @IBMResearch, @RedHat_AI, and @AIatAMD for driving this work!

[sysls @systematicls]

https://t.co/wBaKpAI5Vl

[Atharva @AtharvaXDevs]

Bro just reverse engineered how Codex actually works.
Worth read !!

[Hamza Elshafie @hamzaelshafie]

Wrote an in depth breakdown of Paged Attention and KV cache management in modern inference systems like vLLM.

Starting from first principles:

- LLM training vs inference
- Prefill vs decoding
- Why KV caching exists
- Where memory fragmentation comes from

Then how vLLM style paged KV caching fixes it. Appendix also covers continuous batching, speculative decoding, and quantisation.

Blog: https://t.co/L97njbPDkM

[surya @suryasure05]

gold mine of an account - highly recommend checking out his content!

[Jino Rohit @jino_rohit]

not having a GPU is not an excuse to learn CUDA/triton.

> sign in to platforms like leetgpu/tensara.
> make an account
> solve challenges, look at other solutions , read docs, get better.