Logan Boyd

[Logan Boyd @_loganboyd]

I made a high-level tensor computation language called 𝚒 with a scheduling model that can express numerically stable online blockwise FlashAttention. Here is the full code. (1/)

[Logan Boyd @_loganboyd]

ever uuonder uuhat it uuould be like if uue never did that w shit?

[Logan Boyd @_loganboyd]

@Empty_America imo nicest answer is 24 because you can a nice circular pattern x2, x3, x4, x3. neither of 1x or 5x solutions have this

[Logan Boyd @_loganboyd]

@OpenAI now do P vs NP

[Logan Boyd @_loganboyd]

@GrigoryEvko glad to know I’m not the only one 😭

[Logan Boyd @_loganboyd]

@pau_wee_ doesn’t work on ipad because of scroll

[Logan Boyd @_loganboyd]

@Gowtharr @PalmerLuckey RL

[Logan Boyd @_loganboyd]

If you want to check it out or run the demo yourself, it lives here: https://t.co/UlCMly5wcs.

Curious to hear feedback from compiler folks. Could this sort of scheduling be useful?

(5/5)

[Logan Boyd @_loganboyd]

I made a high-level tensor computation language called 𝚒 with a scheduling model that can express numerically stable online blockwise FlashAttention. Here is the full code. (1/)

[Logan Boyd @_loganboyd]

The hope is that an extremely simple scheduling model makes it easier for humans and search algorithms to find performant schedules.

Current status: working proof-of-concept. Compiler+runtime in Rust, C backend, small Python frontend, zero dependencies. Not fast (yet!)

(4/)

[Logan Boyd @_loganboyd]

@jino_rohit I’m actually working on exactly this with my compiler called 𝚒. it has a really clean scheduling model that can lower the same attention algorithm to “naive” attention or to FlashAttention. https://t.co/UlCMly5wcs

[Logan Boyd @_loganboyd]

@MainzOnX I’ll be looking out for it!

[Logan Boyd @_loganboyd]

@MainzOnX drop part 2 so I can read that too lol

[Logan Boyd @_loganboyd]

@MAstronomers very cool idea. my brain wants so badly for the horizon not to change like so the occlusion of the sky is fixed. I guess that happens only when the camera is polar? or maybe it just needs to point along a longitude line?

[Logan Boyd @_loganboyd]

@yacineMTB ask one

[Logan Boyd @_loganboyd]

@alexocheema this is super cool! how do you compute theoretical peak on a setup like this? is it just based on pure flops or bandwidth/interconnect as well?

[Logan Boyd @_loganboyd]

@__tinygrad__ nvlink on 4090s is a thing, right?

[Logan Boyd @_loganboyd]

@stablechen @ekzhang1 oops, meant to say autodiff gives exact numerical value, not expression 😄

[Logan Boyd @_loganboyd]

@stablechen @ekzhang1 autodiff isn’t quite either of these. whereas numeric diff gives a numerical approximation and symbolic gives an exact symbolic expression, autodiff gives an exact numeric expression only at the point of evaluation. it basically does a big chain rule calculation.