Zihao Ye

[Thien Tran @gaunernst]

Beating CuBLAS was not a goal, but it came out pretty good. I think this is more useful as a concise and hackable "template", rather than being the fastest kernel: bring ur own epilogues, roll a megakernel, ask Codex to fork it. Just like when I first learned Triton.

[Banghua Zhu @BanghuaZ]

Miles + multi-adapter LoRA training, presented by @Osmosis_AI!

[Ruihang Lai @ruihanglai]

Two moments every ML researcher knows. You get onto a new cluster, and week one goes to fitting the framework to your setup, not training. A new architecture lands, and trying it means hacking through a gigantic codebase to stay compatible with the pipeline. What you want to change is small. The code you wade through to change isn't.

This experience is likely not alone, and many researchers we’ve talked to run into similar issues. A year of this on CMU's FLAME cluster left us with one question: what if a framework were built for an agent to adapt and evolve, not just for humans to maintain?

So we introduce PithTrain: a compact, agent-native MoE training system, now ~11K lines of Python, on four principles:

- Compact: fits in one context window
- Python-native: readable tracebacks, no compiled-extension rebuilds
- No implicit indirection: direct calls, each model in its own file
- Agent skills: in-repo playbooks for recurring tasks

Then we measured the thing nobody measures. Same agent, same tasks, only the framework underneath changes: on PithTrain it finishes with up to 62% fewer turns and 64% less GPU time than production frameworks, while training just as fast.

We call this second axis agent-task efficiency, and we believe it deserves to sit alongside training throughput as a metric worth optimizing. Excited to see what people build with it.

Built with amazing collaborators @haok1402, Haozhan Tang, Akaash Parthasarathy, @Zichun_Yu, @junrushao, Todd Mowry, @XiongChenyan and @tqchenml.

Blog: https://t.co/byOKPs9rGQ
Code: https://t.co/AH5ZbwYluV
Paper: https://t.co/hkmDGx9Hc6

[Zihao Ye @ye_combinator]

@madebyollin @CVPR best for your next journey!

[Shuo Yang @Andy_ShuoYang]

Flash-KMeans was only the beginning.

Today, from the Flash-KMeans team, we are releasing FlashLib — a GPU library for fast, predictable, agent-ready classical ML operators.

Up to 26× on KMeans, 19× on KNN, 40× on HDBSCAN, 208× on TruncatedSVD, 47× on PCA, 147× on exact t-SNE, and 49× on MultinomialNB over state-of-the-art (cuML).

Blog: https://t.co/P31SGl0cyT
Code: https://t.co/9nkO2hmeOl

[Luis Ceze @luisceze]

That was a great talk @marksaroufim — started MLsys on an exciting note!

[Yifei Zuo @YifeiZuoX]

For me, the coolest finding is that Muon optimizer is crucial for Parallax to move beyond Softmax Attention.

Lesson — don't evaluate new architectures solely under AdamW, you'll miss the good ones.

paper: https://t.co/yAqClXrJUz
code: https://t.co/D4pgIr1wM7

For the origin of Parallax, check out the LLA paper at ICLR 2026:
paper: https://t.co/85OzoOQlnF
code: https://t.co/eqMYZ0U6qO

[Google Gemini @GeminiApp]

Build your first game with Gemini 3.5 Flash.

Translate everyday objects directly into interactive, digital experiences without complex 3D modeling. Start with a Nano Banana prompt, turn your image into a game in Canvas, and refine your vision for optimal gameplay.

[Lequn Chen @abcdabcd987]

We are obsessed with performance and low level details. Great work by my teammate @xyzw_io . Chat with us if this interests you.

[Yoav Gelberg @yoav_gelberg]

Excited about this new work

As KV compaction becomes increasingly important, we ask whether it’s worth adapting the model itself to perform better under compaction

Turns out, it can really matter

[Tri Dao @tri_dao]

After some mathematical rewrite, turns out all of transformer is a series of gemm + epilogue. Given a few optimized primitives, LLMs (and novice humans) can write speed-of-light kernels for all transformer ops!

[Charles 🎉 Frye @charles_irl]

rolling up to MLSys '26 to meet with @ye_combinator and the winners of our B200 kernel perf competition

quick trip, so i packed a single bag, just my essentials

[Edward Z. Yang @ezyang]

Pro-tip: using CUDA graphs and annoyed that all the kernels have no labels in your profiles? Get a nightly that has mark_kernels context manager: https://t.co/IF3sZhb4S2 (thanks Natalia and Shangdi for implementing!) You need 13.1 driver, but user mode driver is enough

[Mark Saroufim @marksaroufim]

It was an honor to give the keynote at MLSys
Covered how AI systems have evolved, why AI is needed to improve them, why results have disappointed, why the future looks amazing, and why I’m working on this at Core Auto
Recording should be out soon, in the meantime slides

[Tianqi Chen @tqchenml]

#MLSys26 NVIDIA is hosting ice cream social on Friday https://t.co/jFZGXxqtbe after the competition session, looking forward to see folks there

[Edward Z. Yang @ezyang]

Historically, we used https://t.co/uMrrnBoSPF for this. I think this forum is still good for things that are much more about discussion. But actual one big impetus for devlogs as a SSG website https://t.co/wDTjRlq1jK was to make the posts more easily accessible to LLM agents

[Zhijian Liu @zhijianliu_]

Super happy to see my UCSD colleague help port our DFlash to TPU. Big speedup too!! More to come.

[wh @nrehiew_]

This likely means OpenAI does interleaved stages of SFT-RL-SFT-RL rather than the simpler SFT-RL-done pipeline we see with open models

[AI Dance @AI_Whisper_X]

挺有意思的研究。

闭源实验室都对模型规模讳莫如深，但他们其实藏不住模型"知道什么"。而模型知道什么，恰恰就是参数量的指标。
核心逻辑：推理能力可以靠蒸馏压缩到小模型里，事实知识不行。一个模型记得多少冷门事实，直接跟它的参数量挂钩。

知乎博主李博杰为这个写了一篇小论文，构建了一套叫 IKP（不可压缩知识探针）的数据集：1400 个问题、7 层稀有度，扔到 27 家厂商的 188 个模型上跑了一遍，只看事实准确率。

结果在 89 个公开参数的开源模型上，准确率 vs log(参数量) 的拟合 R²=0.917，基本是一条直线。把闭源模型投影上去，规模就估出来了：

GPT-5.5 ≈ 9T
Claude Opus 4.7 ≈ 4T
GPT-5.4 ≈ 2.2T
Claude Sonnet 4.6 ≈ 1.7T
Gemini 2.5 Pro ≈ 1.2T
（90% 置信区间：0.3-3 倍规模）

另外两个发现也挺反直觉：
一是引用数和 h-index 不能预测一个研究者是否被前沿模型认识。两个引用数相近的人，模型给的回答可能完全不一样。它记的是有影响力的工作，不是论文数量。
二是事实容量不会被时间压缩。跨 3 年的 96 个开源模型，IKP 时间系数统计上为零（p<10⁻¹⁵），直接拒绝了 Densing Law 预测的 +0.0117/月衰减。benchmark 在饱和，但事实容量还在随参数继续扩张。

来源：知乎博主 李博杰
侵权联系删
https://t.co/Bt5CiMGc5M

[Edward Z. Yang @ezyang]

A quick survey of PyTorch APIs that allow for controlling precision in a fine-grained way. 🧵