Adam Louly

Inference Engineering @Microsoft

[Adam Louly @LoulyAdam]

Yet another misleading benchmark (batch size of 1).

[Sakura Yuki @sakurayukiai]

At 3,000 tokens/s, your budget per token is 333 microseconds. A basic kernel launch is 4.5µs. Kog hit 3k tok/s on standard GPUs by dropping PyTorch and writing a persistent monokernel in raw assembly. The CPU scheduler is officially the enemy ✨

[Adam Louly @LoulyAdam]

@isbkch Priorities change for sure, I see where you’re coming from, I’ll probably change my mind the moment kids enter the picture too

[Adam Louly @LoulyAdam]

@tenderizzation Bellevue is what SF wanted to be

[Adam Louly @LoulyAdam]

Out of all the MLSYS happy hours, OpenAI’s one was probably the top 1, together AI’s was also dope!

[Adam Louly @LoulyAdam]

@romanbuildsaas That’s azure credits buddy

[Adam Louly @LoulyAdam]

@tenderizzation It is expected.. I guess?

[Adam Louly @LoulyAdam]

@tqchenml Do people have to register even if they’re at mlsys already?

[Adam Louly @LoulyAdam]

@marksaroufim @tqchenml Decent talk, especially the part on how people kept hacking around the leaderboard

[Adam Louly @LoulyAdam]

@danzaria_an @theMadridZone Peak football knowledge for sure

[Adam Louly @LoulyAdam]

I’m at MLSys this week! feel free to come say hi if you want to chat about inference, GPU programming or ml systems

[Adam Louly @LoulyAdam]

@mr_r0b0t @Snixtp Gb10 won’t give you the best results I guess, it does not have optimized nvfp4 tensor cores right? However for that mission you deserve a gb300

[Adam Louly @LoulyAdam]

@mr_r0b0t @Snixtp You can run it on dgx spark too, the gguf version tho

[Adam Louly @LoulyAdam]

@alilakrakbi @isbkch Same

[Adam Louly @LoulyAdam]

@menhguin @cerebras @hud_evals You got the video?

[Adam Louly @LoulyAdam]

@LeeLeepenkman That’s hip not cuda

[Adam Louly @LoulyAdam]

Unrolling a single mfma loop produced 17% perf gain 🤯🤯 I’ll never underestimate the power of unrolling ever again

[Adam Louly @LoulyAdam]

@Simo__Ben @MistralAI @Ministere_TNRA @AmalElfallah @OCPGroup I’m not aware that’s why I asked you, AFAIK things built by our talents has been getting hacked every other week

[Adam Louly @LoulyAdam]

@cyberpengk Most gpu normies don’t know how great AMD GPUs are