Leylin Farlier

Just playing in sandbox.

[Yichi Zhang @YichiZ03]

zero-to-sglang new chapters are out! 📖
Want to read inference engine source code but don’t know where to start?
We’ve just added two chapters to zero-to-sglang, the open-source AI Infra course from SGLang × Datawhale:
📖 2.1 — mini-sglang: What an Inference Engine Looks Like
📖 2.2 — The Journey of a Request + Code Walkthrough

Find us on GitHub: datawhalechina/zero-to-sglang 🫵
#SGLang #LLMInference #AIInfra #OpenSource #LLM

[Marcel Rød @marcelroed]

Introducing the world's fastest tokenizer implementation, Gigatoken!

Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.

These baselines are already multithreaded Rust implementations! 🧵

[Percy Liang @percyliang]

⚡! Tokenization is the first thing you do in CS336 (language models from scratch). If Marcel can work his magic for the rest of the LM pipeline, the world will be a better place.

[Aleph @alephneuro]

We recently obtained the highest-resolution 3D images of the human brain ever taken from outside the skull. This is the first look.

Introducing Aleph, a research lab building brain interfaces for the telepathic future. (1/n)

[Elon Litman @elon_lit]

We developed a unified theory of generalization in deep learning. It explains grokking, double descent, benign overfitting, and implicit bias.

But theory is only half the story. It turns out that optimizing the population risk of any neural network amounts to a small change to your optimizer. 🧵

[Zijie Yan @zijie_y]

Our Megatron Core MoE technical report (88 pages!) is out - probably the most complete systems guide to training MoE at scale. Covers parallelism, memory/comm/compute optimizations, FP8/FP4, and best practices. 1,233 TFLOPS/GPU on GB300 for DeepSeek-V3. https://t.co/3cdJm9FToQ

[Adam Gleave @ARGleave]

We're looking to hire 15+ technical staff this year globally (remote; or in-person in Berkeley, CA or Singapore) to work on core problems in trustworthy & secure AI -- check out roles below 👇

[Massimo @Rainmaker1973]

Man of questionable sanity ✅
Very polite pig ✅
Dancing parrot ✅
Curiously chill chicken✅

[Paul Graham @paulg]

If you resolved to read more this year, here are some particularly pleasing books: Master and Commander, Franklin's Autobiography, The Inimitable Jeeves, My Family and Other Animals, Wing Leader, Some Experiences of an Irish RM, A Time of Gifts.

[ᅠ ᅠ @greenvibe]

I always wonder why the journey is better than the destination..

[chrissy @chrissyykat]

home for thanksgiving and found people are watching the new show pluribus, which is funny timing considering it pairs almost too well with the artificial hivemind work that was just awarded neurips best paper this year

in the show, humanity has turned into a collective hivemind that is perpetually content but only because it’s stripped of individual personality and dissent. the paper warns that reward models are behaving like the virus in the show, sanding away idiosyncratic responses until model diversity collapse

seeing the show’s universal agreement virus after reading the implications section of the paper makes it almost feel like a companion piece lol

[Percy Liang @percyliang]

You spend $1B training a model A.
Someone on your team leaves and launches their own model API B.
You're suspicious. Was B was derived (e.g., fine-tuned) from A?
But you only have blackbox access to B...
With our paper, you can still tell with strong statistical guarantees (p-values < 1e-8).
Idea: test for independence of A's training data order with likelihoods under B.
There are crazy amounts of metadata about training process baked into the model that can't be washed out, like a palimpsest...

[Leylin Farlier @warlockLeylin]

@NVIDIAGeForce GeForce Day

[Nicholas Decker @captgouda24]

This paper is one of the most astonishing feats of sustained data wizardry I have ever seen. Using data from Uber, they are able to estimate the roughness of every road in America and precisely estimate the value people place on it, and so much more. 1/