enbao

gpus! @hazyresearch @stanford

[enbao @enbaoc]

@nash_c_brown Amazing

[Nash Brown @nash_c_brown]

Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon.

Check out the code here: https://t.co/cYtVw0x34r
Shoutout to the FA4 team for the algorithmic innovations and to @stuart_sul for the helpful discussions.

[Stuart Sul @stuart_sul]

(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS!

Alongside this release, we’re equally excited to share some insights we learned while squeezing every last TFLOP out of Blackwell:

(with @hazyresearch & generously supported by @cursor_ai)

[enbao @enbaoc]

wrote a personal piece on the importance of side quests and the point of it all -

https://t.co/vsMFXmPs0r

[enbao @enbaoc]

@oggyxe thank you!

[enbao @enbaoc]

my first blogpost related to GPUs! this one looks at pyutils, a small but important part of the ThunderKittens library that allows kernels to be launched with PyTorch. https://t.co/igBv4WRDlk

[enbao @enbaoc]

@darrowoah yes, I’ll try to asap! I heard it’s great

[enbao @enbaoc]

@GenixCS thanks!

[enbao @enbaoc]

@EitanTurok thanks!

[enbao @enbaoc]

@jashvira thank you!!

[enbao @enbaoc]

@raayandhar will check this out! seems very relevant

[enbao @enbaoc]

@monty10x thank you!

[enbao @enbaoc]

@nathanrs thanks so much, yours is amazing too!

[enbao @enbaoc]

this is also my first real technical post, so feedback is very welcome!

[enbao @enbaoc]

fell off my bike and injured both my legs to force myself to sit at my desk and hashtag lock in

[enbao @enbaoc]

keep it fun

[enbao @enbaoc]

@seatedro https://t.co/ORZg66AUsv!

[enbao @enbaoc]

wrote a piece on the things i've missed back home!
https://t.co/1ecYpjSfjU

[enbao @enbaoc]

@paularambles love love ushi taro

[enbao @enbaoc]

“we are squarely an ‘age of research’ company” hell yeah

[Dwarkesh Patel @dwarkesh_sp]

The @ilyasut episode

0:00:00 – Explaining model jaggedness
0:09:39 - Emotions and value functions
0:18:49 – What are we scaling?
0:25:13 – Why humans generalize better than models
0:35:45 – Straight-shotting superintelligence
0:46:47 – SSI’s model will learn from deployment
0:55:07 – Alignment
1:18:13 – “We are squarely an age of research company”
1:29:23 – Self-play and multi-agent
1:32:42 – Research taste

Look up Dwarkesh Podcast on YouTube, Apple Podcasts, or Spotify. Enjoy!