Vijay

Systems and GPU Performance Mechanic - TBD

Ex. CUTLASS 3.x / 4.x etc

[Vijay @__tensorcore__]

As of last week, I am no longer at NVIDIA 🧵

Leaving the CUTLASS team was extremely hard. I will dearly miss my incredible colleagues and the extremely compelling mission statement of creating the world's best accelerator programming model w/ hardware software codesign 💚

[Tri Dao @tri_dao]

After some mathematical rewrite, turns out all of transformer is a series of gemm + epilogue. Given a few optimized primitives, LLMs (and novice humans) can write speed-of-light kernels for all transformer ops!

[Han Guo @HanGuo97]

LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels.

CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip.

Bonus: LLMs can write fast CODA kernels too (approaching SoLs).

[Jack Zhang @jcz42]

We built a kernel abstraction to rewrite the entire transformer stack as GEMM + Epilogue kernels!

Neural net architectures such as transformers consist entirely of matrix multiplications and elementwise nonlinearities such as RMSNorm, log sum exp, and gated activations. Fusing these elementwise nonlinearities into GEMMs in both the forward and backward passes allows us to make training and prefill as compute-bound as possible!

Our kernel abstraction CODA is implemented in CuTeDSL, and by abstracting away the fixed prologue and main loop of the GEMM kernel, we expose an epilogue function where LLMs like Claude can easily implement elementwise nonlinearities in fusions approaching speed-of-light!

[Perplexity @perplexity_ai]

We’ve developed our own inference engine Runtime-Optimized Serving Engine (ROSE) to serve models ranging from embeddings to trillion-parameter LLMs.

With CuTeDSL integrated into our inference engine, Perplexity can build the specialized GPU kernels faster to bring models up to peak performance on NVIDIA Hopper and Blackwell GPUs.

[Vijay @__tensorcore__]

@cHHillee @tenderizzation One could say you’re still a junior celebrity

[Vijay @__tensorcore__]

@cHHillee @tenderizzation Learning to be a celebrity I see!

[resham ☻ @Reshusaur]

new walk of shame:
agent still working, but the cafe closed

[Vijay @__tensorcore__]

@PatrickToulme @AlpinDale This ain’t true. You have the nvvm dialect for native PTX authoring too without “escape hatches”

[tender @tenderizzation]

[ENG SUB] how it feels to use eager pytorch in 2025

[Alex Zhurkevich @cudagdb]

Tomorrow: Blackwell Programming lecture by yours truly at Stanford CME213, Gates B3, 1:30–2:50 PM.

Bring sharp questions.

[Kimi.ai @Kimi_Moonshot]

We're open-sourcing FlashKDA — our high-performance CUTLASS-based implementation of Kimi Delta Attention kernels. Achieves 1.72×–2.22× prefill speedup over the flash-linear-attention baseline on H20, and works as a drop-in backend for flash-linear-attention.

Explore on github: https://t.co/sf4UohXDWY

[Yuchen Jin @Yuchenj_UW]

Meta released Avocado, they call it Muse Spark.

It's not open source (a bit sad).

Meta TBD lab rebuilt the entire pretraining stack in 9 months and reached similar capability with >10x less compute than Llama 4 Maverick.

I still think infra is the real moat in AI labs. You can train models much faster with a good infra, and it allows researchers to experiment with many more ideas much more quickly.

[Shengjia Zhao @shengjia_zhao]

Excited to share what we’ve been building at Meta Superintelligence Labs! We just released Muse Spark, our first AI model. It's a natively multimodal reasoning model and the first step on our path to personal superintelligence. We've overhauled our entire stack to support scaling, and this is just the beginning.

https://t.co/KNVjgMcch1

[Alexandr Wang @alexandr_wang]

1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵

[Ji-Ha @Ji_Ha_Kim]

Very cool! I worked on this recently, and I actually used an identical approach early on. But I believe there is a significantly better approach - a **single** minimax rational iteration can beat 5 polynomial steps!

[Alex Zhurkevich @cudagdb]

Trtllmgen kernels are now open. Fastest prefill and decode kernels for our target workloads. We wrote these to win InferenceX, MLPerf, other benchmarks. Powering some of today’s top served models. Dive in, learn, use them, or level up your own. Enjoy.

https://t.co/2aQBwcdnZL

[Edward Z. Yang @ezyang]

In my opinion, here are the most important ideas of CuTe Layouts (https://t.co/7ZSFcDeiqt) 🧵

[Tri Dao @tri_dao]

The frontier has increasingly shifted to hybrid models - from Qwen to Kimi-Linear and now with NVIDIA's Nemotron-3 Super - that rely on a strong linear sequence model. Today we release Mamba-3, the most powerful linear model to date.

https://t.co/OpMmqEWMkP

[Vijay @__tensorcore__]

https://t.co/rkvPtJMmbe

[Anne Ouyang @anneouyang]

Excited to share @Standard_Kernel's seed round and some reflections on what we’ve learned about kernel generation and what we believe is next. Grateful to our amazing team, supporters, and the broader community pushing this space forward.