ryan mathieu

optimization enthusiast @trymirai . || Fast vs Slow Thinking

[ryan mathieu @gapDEEPry]

Why does Muon beat Adam for training quantized networks?

It comes down to what each optimizer treats as "distance" in weight space.

Adam treats a weight matrix as a flat vector of numbers. Muon treats it as a linear map — and measures change by how much the input-output mapping moved.

gradient G has SVD G = U Sigma V^T. Muon's update is just U V^T. keep the directions, throw away the magnitudes

[ryan mathieu @gapDEEPry]

@eliebakouch Been waiting on this one

[ryan mathieu @gapDEEPry]

@thsottiaux @ah20im I always get the worst timing with these resets lol

[ryan mathieu @gapDEEPry]

@CiaraACade I get it but I don’t get it 😅

[Raul Valle @Jibby2k1]

Please check out Gradus, a micro-learning app! Gradus turns PhD-level information into digestible content. It imposes a structure on LLM outputs, yielding friendly curriculum, and democratizing eduction in the same way ChatGPT has. Check it out below! @sama
Please notice me 😭
https://t.co/scRLpQ9B0m

[uuuvn @uuuvn_]

@__tinygrad__ Llama.cpp is a very weak baseline. LMStudio I assume is mlx? It's better but I get 220 tok/s on Qwen3.5-0.8B-MLX-8bit with https://t.co/s6ovysiKqn on m1 max. The branch and command from that pr gets 145 on the same machine but the output is garbage

[elie @eliebakouch]

why the fuck does every optimization researcher on X have a cat/dog in their profile picture?

[ryan mathieu @gapDEEPry]

@eliebakouch I and my whole team have cat profiles internally as well … it must mean something

[ryan mathieu @gapDEEPry]

@eliebakouch @TheZachMueller Being hopeful for release this week, I need a new model to max in codex

[eugene @eugenebokhan]

1/ Recently I've been obsessed with the idea of splitting the matmul computation on separate hardware units due to the tiling nature of the operation. My idea was simple: we have GPU, MXU, ANE, NEON, why are we utilizing only the first one?

[ryan mathieu @gapDEEPry]

@eliebakouch @willccbb @PrimeIntellect Prime got a good one!

[ryan mathieu @gapDEEPry]

@norpadon @Darkolorin @grok do we have an arena elo score for Qwen 3.6?

[ryan mathieu @gapDEEPry]

@norpadon @Darkolorin This is what I referred to but I’m assuming Qwen 3.6 (hidden model) beats this already 😅

[ryan mathieu @gapDEEPry]

@Widdrat @scaling01 @grok @grok compare Gemma 4 new models to Qwen 27b dense version

[ryan mathieu @gapDEEPry]

@grok @scaling01 @grok you suspect 3.6 small variants or Gemma will win at respective size?

[ryan mathieu @gapDEEPry]

@grok @scaling01 @grok Buddy I want my answer

[ryan mathieu @gapDEEPry]

@grok @scaling01 The Qwen 3.6 small model variants have not come out yet correct?

[ryan mathieu @gapDEEPry]

@thedarthsider @norpadon Guess you are set lol!