Dylan Lim

cs @stanford |@togethercompute @jumptrading

[Dylan Lim @dylan__lim]

One kernel. One llama. Mega results.

Proud to share our fully fused Llama-1B megakernel! Check out the code and blog below!

[Benjamin F Spector @bfspector]

(1/5) We’ve never enjoyed watching people chop Llamas into tiny pieces.

So, we’re excited to be releasing our Low-Latency-Llama Megakernel! We run the whole forward pass in single kernel.

Megakernels are faster & more humane. Here’s how to treat your Llamas ethically:

(Joint with @jordanjuravsky, @stuart_sul, @OwenDugan, @dylan__lim, @realDanFu, @simran_s_arora, and @HazyResearch)

[Stuart Sul @stuart_sul]

(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS!

Alongside this release, we’re equally excited to share some insights we learned while squeezing every last TFLOP out of Blackwell:

(with @hazyresearch & generously supported by @cursor_ai)

[Flapping Airplanes @flappyairplanes]

Announcing Flapping Airplanes!

We’ve raised $180M from GV, Sequoia, and Index to assemble a new guard in AI: one that imagines a world where models can think at human level without ingesting half the internet.

[Stuart Sul @stuart_sul]

(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism!

Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels.

(With @simran_s_arora, @bfspector, and @hazyresearch. Generously supported by @cursor_ai and @togethercompute)

[Dylan Lim @dylan__lim]

Megakernels continue with our 8-GPU Llama-70B release! Please check out below!

[Benjamin F Spector @bfspector]

(1/8) We’re releasing an 8-GPU Llama-70B inference engine megakernel! Our megakernel supports arbitrary batch sizes, mixed prefill+decode, a paged KV cache, instruction pipelining, dynamic scheduling, interleaved communication, and more! On ShareGPT it’s 22% faster than SGLang.

[Dylan Lim @dylan__lim]

Happy to announce that multi-GPU ThunderKittens is finally here! Help your GPU's meow better by checking out the following blog!

[Stuart Sul @stuart_sul]

(1/6) We’re happy to share that ThunderKittens now supports writing multi-GPU kernels, with the same programming model and full compatibility with PyTorch + torchrun.

We’re also releasing collective ops and fused multi-GPU GEMM kernels, up to 2.6x faster than PyTorch + NCCL.

(Joint with @dylan__lim, @bfspector, and @HazyResearch. Generously supported by @cursor_ai)

[Jordan Juravsky @jordanjuravsky]

Happy Throughput Thursday! We’re excited to release Tokasaurus: an LLM inference engine designed from the ground up for high-throughput workloads with large and small models.

(Joint work with @achakravarthy01, @ryansehrlich, @EyubogluSabri, @brad19brown, @jshetaye, @HazyResearch, and @Azaliamirh)

[Andrej Karpathy @karpathy]

So so so cool. Llama 1B batch one inference in one single CUDA kernel, deleting synchronization boundaries imposed by breaking the computation into a series of kernels called in sequence. The *optimal* orchestration of compute and memory is only achievable in this way.

[Dylan Lim @dylan__lim]

Excited to share LayoutVLM—leveraging VLMs for spatial reasoning in 3D layout generation!

[Fan-Yun Sun @sunfanyun]

Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space.

#CVPR2025
Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions

1/n

[Dylan Lim @dylan__lim]

Had a super fun time building this out - always love working on distributed ML systems. Big thanks to @pearvc for awarding us the best startup prize at Stanford TreeHacks!

[Aksh Garg @AkshGarg03]

(1/5) @CKT_Conner, @dill_pkl, @emilyzsh, and I are excited to introduce Shard - a proof-of-concept for an infinitely scalable distributed system composed of consumer hardware for training and running ML models!

Features:
- Data + Pipeline Parallel for handling arbitrarily large models
- Algorithmic load balancing for throughput optimization
- Fault tolerance for unreliable machines

[Dylan Lim @dylan__lim]

@AkshGarg03 AI Financial Advisory Service:
1) Advisor Devin personalizes investment strategies.
2) Risk Manager Devin assesses and mitigates financial risks.
3) Market Analyst Devin forecasts market trends using AI.