Today’s models and algorithms evolved in the context of the kernels that run them - the most incredible nights of my life have been spent breaking those shackles with the Kittens Virtual Machine.
The MFU that we have been able to squeeze out of GPUs, and the research directions that we have been taken down, are unprecedented in the space of model–hardware co-design. Reach out if you're interested :)
We’ve developed the world’s most complete megakernel framework. It can perform entire training runs, serve inference, play video games, and even do all three simultaneously in a single kernel.
We’ve spent the past year developing new primitives for GPUs. We believe this enables new algorithms to change how machines learn, open new markets, and dramatically reshape AI’s cost structure.
We have raised a $1 billion Series A, co-led by Index and Lux.
New scoop from me: @flappyairplanes is now valued at $6B with a new $1B funding round, and they’ve been working on ways to make AI more efficient (including an effort to squeeze more performance out of investor @nvidia’s GPUs). Gift link below. 🍦
https://t.co/avoTPOroQH
(1/5) Great to be at @sequoia to give a sneak peek of one of our research directions!
TL;DR one path to data-efficiency may be to “abuse GPUs like they’ve never been abused before”