Laura Wang

[Ty Geri @ty_geri]

update the management books

[Bing Xu @bingxu_]

This may be one of the first real signs of superhuman intelligence in software. On some of the most optimized attention workloads, agents can now outperform almost all human GPU experts by searching continuously for 7 days with no human intervention inside the optimization loop.

Terry and I started agentic coding efforts at NVIDIA 1.5 years ago. Neither of us knew GPU programming, so from day one we pushed toward fully automated, human-out-of-the-loop systems. We call it blind coding.

Over those 1.5 years, the two of us generated 4 generations across 2 agent systems. Since the 2nd generation, the stacks have been self-evolving. Each agent is now around 100k non-empty LOC.

When we released the blind-coding framework VibeTensor in January, the implication was easy to miss. AVO makes the signal clearer.

My bet is: blind coding is the future of software engineering. Human cognition is the bottleneck.

[PyTorch @PyTorch]

Building on the previous correctness-focused pipeline, KernelAgent can now integrate GPU hardware-performance signals into a closed-loop multi-agent workflow to guide the optimization for Triton Kernels. Learn more: https://t.co/r2WqASIhWG @KaimingCheng @marksaroufim

[Elon Musk @elonmusk]

@cb_doge I’m started off knowing nothing about rockets, satellites, cars, etc, but I learn fast

[AK @_akhaliq]

Nvidia presents VibeTensor

System Software for Deep Learning, Fully Generated by AI Agents

[Mark Saroufim @marksaroufim]

Pretty interesting Kernel LLM result, the beginnings of a new fast CuteDSL kernel zoo that's Quack inspired called Oink!

An AI generated fused RMS norm kernel was integrated into VLLM and is showing 40% speedups relative to the existing RMS norm kernel in VLLM and an e2e 1.6% over the entire system https://t.co/1Kz3z6vN7z

Looking at the kernel specifically and comparing it vs the Quack one is also interesting https://t.co/OqFH4RQ91d

First off, the code is much longer than Quack's and that's because the AI tries to effectively write a mini heuristic autotuner that's splatted over the file - so for instance Deepseek has a hot shape of 7168, at bf16 and if we choose to copy 256 bit vectors we get 16 vector elements. 7168 / 16 = 448 vectors across the row so we can just choose 224 threads per row to get 448 / 224 = 2 vectors per thread. This would be quite tedious for humans to work out per shape especially without a long running autotuner.

This doesn't always work perfectly and the AI admits that this can segfault if used in conjunction with cluster launches and direct GMEM loads which brings me to the second cool trick.

The AI figured out it could do direct_gmem which skips smem for data staging but keeps it for reduction

The kernel is still quite long at 3K LOC but I suspect this can be brought down significantly, for instance the AI built its own tensor marshalling abstraction when it could have just leveraged tvm-ffi but overall it seems good at taking existing code written by experts such as Quack and modifying it to make a bit faster by using more tricks

Using VLLM as an eval suite is quite nice, I'm not sure on where we'll converge as a community with excessive fallbacks on hard to fully test kernels, I suspect those will make stability and/or determinism work much more challenging but between this work and the work the FlashInfer team is doing on using full systems as an eval suite, I'm more optimistic we'll up end up with SOTA AI kernels this year.

[Laura Wang @LaurawlyLaura]

KernelFalcon: Agentic kernel authoring is getting very real.
vLLM: https://t.co/F6DcSSQmPR
PyTorch: https://t.co/ut2NLAHSdi

[Elliot Arledge @elliotarledge]

This is my favorite clip of the new Elon pod. He opens up saying xAI struggles with memory usage/bandwidth and CUDA kernel optimization (matmul, attention, MoE, etc). If you are good kernel or performance engineering in general, you should apply. Steer the world in a better direction.

[Laura Wang @LaurawlyLaura]

@elliotarledge @PyTorch Perf is next—stay tuned~

[PyTorch @PyTorch]

KernelFalcon achieves 100% correctness across all 250 KernelBench L1–L3 tasks through a deep agent architecture that structures the problem instead of prompting harder.

The system combines hierarchical task decomposition, deterministic orchestration, grounded execution, and parallel verification to generate GPU kernels that compile to PTX, execute on real hardware, and preserve PyTorch semantics.

💡Read our latest blog from @LaurawlyLaura and collaborators at Team PyTorch:
https://t.co/AwH0dOFxET

#PyTorch #KernelFalcon #AIInfrastructure #OpenSourceAI

[Future 42 @future42org]

As WA State Education Spending goes UP, Reading and Math scores go DOWN👎

Without question, the Return On Investment for education in Washington State is getting worse.

Link
https://t.co/N5LjpMnMuD

Via @EdunomicsLab

[Laura Wang @LaurawlyLaura]

Introducing NNVM Compiler: A New Open End-to-End Compiler for AI Frameworks https://t.co/ohSannghzj via @awscloud

[Tianqi Chen @tqchenml]

Introducing NNVM compiler https://t.co/Qvju6xevBO brings MXNet, PyTorch, Caffe2, CoreML to bare-metal hardware backends with #TVM stack https://t.co/XjGmstuhT8

[Tianqi Chen @tqchenml]

I am proud to announce TVM, our new deep learning compiler toolchain https://t.co/lIPoX0FPXL https://t.co/KsPmVEF1pg @guestrin @uwcse

[Mark Harris @harrism]

What a cool award! I've always been proud of this paper. @jowens #CUDA

[NVIDIA @nvidia]

Tune in now to watch NVIDIA CEO Jen-Hsun at the NVIDIA Special Event! #GameReady https://t.co/SjipOmfUrj

[Laura Wang @LaurawlyLaura]

VR village is pretty cool at #GTC16 ! Live Finding Nemo creation by a Pixar artist.

[Mark Harris @harrism]

CUDPP 2.2 released: with new suffix array primitive, faster BWT/compress, bug fixes. Thanks @jowens team! http://t.co/pGPgt8iAnA