Sasha Krassovsky

Performance @AnthropicAI

[Sasha Krassovsky @bztree]

@beffjezos Is that a computationally irreducible F-117 Nighthawk?!

[Sasha Krassovsky @bztree]

@effectfully yes dunno = 50% no but more aggressive I think

[Sasha Krassovsky @bztree]

@beffjezos You lose 100% of the shots you don’t take

[Sasha Krassovsky @bztree]

Opus 4.7 haters in shambles

[David Hershey @DavidSHershey]

Many (many) months later, Claude is finally challenging the Elite Four and will likely become a Pokemon Champion tonight 🥲

https://t.co/5ekTUqK36p

[Sasha Krassovsky @bztree]

@i2cjak My timeline seems to disagree with you

[Sasha Krassovsky @bztree]

@ThePrimeagen What is C? I only know ⌘ and ⌃. ⌘-c and ⌘-v works fine for me in Terminal-dot-app

[Sasha Krassovsky @bztree]

This is the part of today’s announcement I’m most excited about 🙂

[xAI @xai]

SpaceXAI and @AnthropicAI have also expressed interest in partnering to develop multiple gigawatts of orbital AI compute capacity

[Sasha Krassovsky @bztree]

@benhylak @pronounced_kyle I just finished the part where he puts number theory into a formal system, around page 230. It’s getting tough to keep going but I’m on a mission

[Sasha Krassovsky @bztree]

@cmuratori "Loading" and "updating the screen" don't seem all that related anyway? Like you can have a game giving you a load screen and updating the loading spinner at 60 FPS if it's really loading GBs from your hard drive.

[Sasha Krassovsky @bztree]

@awesomekling EU nutrition labels drive me nuts. The per 100g system is obviously inferior. Like suppose I want to have a protein bar. I care how much is in the protein bar, not in a glob of 2.37 protein bars

[Sasha Krassovsky @bztree]

@filpizlo @zuhaitz_dev My mind was blown reading the C++ FQA for the first time. C++ really is just reinventing every C feature in its own way.

[Sasha Krassovsky @bztree]

OK last post for the night: I tried all the fancy stuff they recommended in their GEMM doc: Z-curve, static extents, accumulation group synchronization. None of it seemed to make any performance improvement - I seem to be stuck at 40 TFLOPs in bf16 across a variety of shapes.

[Sasha Krassovsky @bztree]

I got my M5 MacBook over the weekend and had some time to mess around with Metal 4 and the Neural Accelerators!
Wanted to document some of my first impressions below:

[Sasha Krassovsky @bztree]

@__simt__ @anemll @ekryski @mweinbach How do you use the fp19? When I had my metal kernel mark the inputs as `float`, the profiler seemed to tell me it wasn't using the neural accelerator, but the normal fp32 ALUs?

[Sasha Krassovsky @bztree]

I got my M5 MacBook over the weekend and had some time to mess around with Metal 4 and the Neural Accelerators!
Wanted to document some of my first impressions below:

[Sasha Krassovsky @bztree]

@ekryski Code here - you can run `uv run test_kernels.py`. Make sure you set `MTL_CAPTURE_ENABLED=1`!

https://t.co/aTAgCCRgzb

[Sasha Krassovsky @bztree]

@mweinbach I believe that’s for allocating Tensors on the host, nothing to do with the neural accelerator

[Sasha Krassovsky @bztree]

@mweinbach Yes! It’s a very good document. I haven’t implemented the Z-order traversal they recommend yet, but plan to.

[Sasha Krassovsky @bztree]

@__alpoge__ @gallabytes 🪵🪵🪵

[Sasha Krassovsky @bztree]

Overall had a fun time! To close off with some criticisms:
- it took me a long time to figure out how to enable Metal 4. I wish this were better-documented
- MPP seems a little boiler-platey. I wish there were a slightly more convenient syntax for this stuff, but not a dealbreaker.
Hope this was interesting!

[Sasha Krassovsky @bztree]

I was also expecting a much more dramatic speedup from the Neural Accelerator. It seemed that with my original tile size of 32x32, I was only getting 244 GB/s of memory bandwidth. Bumping it up to 64x64 gave me 740 GB/s, dropping the time to 3.36ms!