Henry Wildermuth

[Stuart Sul @stuart_sul]

(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs.

Joint work with @nash_c_brown, @hmwildermuth, @tmwilliamlin168, and @ellev3n11