Ologan

ADHD mutant
Former academic failure

all-in NVFP4 w/ CuTe DSL

[Ologan @OloganAI]

@henrylhtsang Preach.

Always start with the dumbest model to think ideas through. Then level up.

You stay sharp and avoid trusting LLMs too much.

Dumbest models are *technically* capable of matching the best model's output with proper input. Pre-reason.

And they are cheap and *fast*.

[Ologan @OloganAI]

@suchenzang Duke doesn't look like an outlier tbf

[Iaroslav Elistratov @iaro_e]

Build B200 attention kernel from scratch in CUDA and a little PTX, starting from a naive baseline and reaching 94.4% of FlashAttention-4.

I spent the last couple of months building visual, beginner-friendly progression around 60 diagrams.

We first develop an intuitive understanding of how the naive baseline works, then add one major optimization at a time, pairing each step with detailed diagrams, concise explanations, and code.

For the capstone project, we plug the final kernel into a video-generation model.

https://t.co/YFT7n2K6LX

[Iaroslav Elistratov @iaro_e]

Open to kernel research / ML systems roles (training and inference optimization, performance engineering). Fully remote. DM me, happy to chat

[rohan anil @_arohan_]

What would be great is if @SchmidhuberAI could weigh in on the recurrent depth discussions.

[Ben Koska @BenKoska]

don't believe big serverless when they say you can't have NCU in your containers while maintaining security boundaries. It only costs you sanity (but who needs that anyway)

[Ologan @OloganAI]

@Dorialexander The Grifting is strong with this one:

https://t.co/1f5cEIuG6Y

[Ologan @OloganAI]

EU leadership should do a shake-up but they won't/can't/don't care.

The consequences down the road will be 10x worse for them.

What a waste of potential.

[Trent Kelly @TrentKelly2472]

@MultiverseCompu @ArtificialAnlys I'm sorry man it's just so over for euros

[Ologan @OloganAI]

@Dorialexander EU AI sovereignty was dead before that.

ES/IT/DE blocked FR’s AI DC initiative because they depend on their ⚡️⚛️

ES still blocking home ⚛️ generation in spite of the blackout and VdL”s orders.

Cancerous bureaucracy.

[Ologan @OloganAI]

@thsottiaux Sol (today): closing plan because 1.235 > 1.234.
Me: Hang on, can you improve that?
Sol: oh sure, totally, easy peasy
Me: then fix it
Sol: I need explicit authorization
...

[Ologan @OloganAI]

Sol quite nerfed today before Astra release.

Hours chasing perf problem. I interrogate it, back & forth… it added a "compatibility layer" to Flash Attention. Dumber than usual.

I can't wait, got deadlines.
$200/m
If no fix, I'll have to switch back to Ant tom🤷‍♂️

@thsottiaux

[Ologan @OloganAI]

@thsottiaux OK, I'm not crazy.

https://t.co/KoHi33lX9f

> Do yourself a favour and just take the next 2-3 days off, this happens every time before a new model release.

But see, some of us can't take 3 days off.

[Ologan @OloganAI]

@henrylhtsang I wonder if ChatGPT Pro (web) was made purposefully bad at rendering long conversations to force us to restart.

[Ologan @OloganAI]

@Dorialexander How are the limits lately?

If it's good at writing GPU kernel code I'll switch back tonight. Sol became useless.

[Ologan @OloganAI]

@jbhuang0604 My fav youtuber

[Jia-Bin Huang @jbhuang0604]

Speculative Decoding is the coolest trick for speeding up LLM inference!

Check out the video and learn why rejection sampling preserves quality, and how methods such as draft trees, Medusa, MTP, EAGLE, and DFlash further accelerate LLM inference.

https://t.co/uxwHsSdHM6

[Ologan @OloganAI]

Do we know yet how good is Fable 5.1 on gpu kernel benchmarks?

Kernelbench* seem dead. @nvidia compute-eval seems still alive.

[MikaStars★ @MikaStars39]

The best peft paper I've ever read this year.
https://t.co/nSmbrrwjUg

[Ologan @OloganAI]

Model research/training is dominant 16-bit GEMM in the backward (even for 4-bit weights).

Try on @modal before you spend $5k on fancy GPUs (+ $200-400/yr bills @ 600w).

[Thien Tran @gaunernst]

At 600W, 5090 is ~10% slower than PRO 6000.
(except the nerfed BF16 and FP8 😭)

[Ologan @OloganAI]

@Leik0w0 @nvidia @gaunernst @modal @vast

[Ologan @OloganAI]

Don't waste your money on nerfed @nvidia SM120:

- tiny smem, no tcgen05/tmem
- 16-bit/8-bit GEMM capped 50% (ht/ @gaunernst)
- max cluster (1, 1, 1)

I gave up: local dev on $350 5060 8GB + jobs on @modal ($)/@vast ($$).

Real blocker is no ncu on cloud SM100, anyway.

[Lazarz @Laz4rz]

Owning a 5080 and buying a 5090 is probably the single most shared experience