Kirill Acharya

PhD student @Stanford

[Kirill Acharya @kirillacharya]

Super excited Matryoshka Attribution is finally out! 🪆

MAttr learns a nested ranking of model components across sparsities, directly optimizing intervention performance.

#1 on MIB by a wide margin, transfers across tasks and extends to parameter attribution!

Really enjoyed working on it with @aryaman2020, @NathanHu12, @StevenyzZhang, @noahdgoodman, @jurafsky, @ChrisGPotts

[Aryaman Arora @aryaman2020]

New paper! 🫡

We introduce Matryoshka Attribution, a new attribution method which uses gradient descent to find which parts of a neural network are responsible for a behaviour.

MAttr is #1 on the Mechanistic Interpretability Benchmark by a wide margin (2.9× the runner up).

[Kirill Acharya @kirillacharya]

@karinanguyen @jakubzegzulka @karinanguyen very impressive! I believe this is the right direction to interact with information, and not old-school static ways. It can also be applied to the papers, articles and the rest. Exciting!

[noahdgoodman @noahdgoodman]

how should we find representations in neural nets? how about the way we find everything else these days — gradient decent!

[Kirill Acharya @kirillacharya]

@aryaman2020 🪆

[Kirill Acharya @kirillacharya]

@MaikaThoughts Great article, @MaikaThoughts! A very useful prism to look through at the skills for launching successful products

[Kirill Acharya @kirillacharya]

Great job, @StevenDillmann!

[Steven Dillmann @StevenDillmann]

We're releasing Terminal-Bench-Science: a benchmark for evaluating AI agents on research workflows across scientific domains.

An ongoing Stanford-led community effort, built by the team behind Terminal-Bench together with scientific domain experts at research institutions worldwide. v0.1 has 70 tasks. Claude Opus 5 solves only ~30%.

1/n 👇

[Kirill Acharya @kirillacharya]

Congrats, @sanmikoyejo!

[Sanmi Koyejo @sanmikoyejo]

Grateful to be included in @TIME's 2026 TIME100 AI.

This recognition reflects work on AI evaluation, measurement, safety, and security that isn't mine alone: the students and postdocs at @stai_research; collaborators across @StanfordAILab and @StanfordHAI; the @VirtueAI_co team, whose work continues elsewhere; and the colleagues who reliably tell me when an evaluation doesn't hold up.

Congratulations to everyone else on this year's list, and thanks to TIME.

#TIME100AI
https://t.co/wxEvPy3oSp

[Kirill Acharya @kirillacharya]

@gegelz 💪

[Peter Richtarik @peter_richtarik]

Great event; see you in Sydney!

[Tilde @tilderesearch]

Introducing Online KL Shampoo (OKLS), an optimizer that brings a KL-optimal approximation of full-matrix AdaGrad to language-model training.

Diagonal optimizers ignore correlations between gradient coordinates. Full-matrix AdaGrad captures this geometry but requires quadratic state. Muon considers correlations but not their history. OKLS closes this gap using KL-optimal Kronecker factors, whitening matrix gradients across both row and column directions while remaining naturally scale-invariant.

The main challenge is computing fresh inverse-square-root preconditioners at every step. Even one-step staleness can destabilize training. We make zero-staleness preconditioning practical with Scaled CANS Coupled Newton–Schulz: 10 iterations, 27 FP16 GEMMs, and FP32 accumulation.

OKLS achieves 1.45× the parameter efficiency of Muon while retaining 98% of its training throughput. Across 200M–1B models, an OKLS model matches a Muon model roughly 1.5× larger.

[Rustem @ruuustem_10]

Hi everyone! I’m looking for internship positions in industry to gain a valuable experience. I’m particularly interested in internships with a focus on optimization aspects of LLM pre-training/post-training, where I believe my research experience can be useful

[Kirill Acharya @kirillacharya]

Great work @marcelroed!

[Marcel Rød @marcelroed]

Introducing the world's fastest tokenizer implementation, Gigatoken!

Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.

These baselines are already multithreaded Rust implementations! 🧵

[Kirill Acharya @kirillacharya]

@StevenDillmann let's go @StevenDillmann!

[Steven Dillmann @StevenDillmann]

🚨 Shout-out to Terminal-Bench Science at the #ICML2026 AI for Science Conference today!

Thanks to the @AI_for_Science organizers @wellingmax, @marinkazitnik, @MengdiWang10, @SherryLixueC, @sungsoo_ahn_, Yixuan Wang, @mia_rosenfeld for the feature & @allen_ai's CEO Peter Clark for the picture :)

[Kirill Acharya @kirillacharya]

@BhathalTanvir0 @icmlconf Thanks Tanvir!

[Kirill Acharya @kirillacharya]

Had a lot of fun presenting CertJudge at @icmlconf ICML 2026 AI4Math and Deep Learning for Code Workshops on July 10-11. Thanks to all authors and listeners!

[Ethan Hersch @EthanHersch]

🚨🚨Announcing CertJudge🚨🚨

But who judges the judge? …

New work from Stanford University, Harvard University, Hong Kong Baptist University

[Kirill Acharya @kirillacharya]

@MaikaThoughts Love hiking! 📈

[Jessica Chudnovsky @jchudnov]

Presenting “Internal Data Repetition Destroys Language Models” today as an oral at the Foundations of Deep Generative Models workshop (#ICML2026)! Hope to see you there.

⏰ 11:15am Friday July 10
📍 Conference room S 318

Also, some exciting news: we're grateful and honored to have received the workshop's Runner-up award!

With @JoshuaK92829, Noam Levi*, @RylanSchaeffer, @yegordb, @sanmikoyejo, and David Donoho