lumina-davhide

Building at the intersection of ML & DeFi. | Shilling high-utility gems & $SOL plays. | DM for strategies & collab. 🚀

[Vuk Rosić 武克 @VukRosic99]

Muon optimizer (now used by probably EVERY AI lab)

1. Take the gradient G of a hidden weight matrix W (G = how the loss wants to nudge the weights W).
2. Add momentum: B = mu * B + G (B = a running average of past gradients; mu = momentum decay, about 0.95).
3. Orthogonalize B with 5 Newton-Schulz steps to get O (O = the orthogonalized update: a quintic polynomial applied to B that pushes every singular value of B toward 1).
4. Update the weights: W = W - eta * O (eta = the learning rate; step W against O).
5. Use Muon only for 2D hidden layers - keep AdamW for embeddings, the output head, and biases.

The trick: a momentum update is nearly low-rank - a few directions dominate and drown out the rest. Orthogonalizing flattens all singular values to about 1, so rare-but-useful directions get equal weight. It is steepest descent in spectral norm instead of Euclidean - and it stores one momentum buffer, half of Adam's memory.

Pros:
- about 2x compute efficiency vs AdamW at scale (same loss at 0.519x FLOPs, Moonlight scaling laws).
- half the optimizer memory of Adam (one buffer, not two).
- Newton-Schulz runs in bf16 on GPU - cheap, no SVD.
- already scaled to production (Moonlight 16B MoE / 5.7T tokens, Kimi-K2).

Cons:
- only for 2D hidden layers - embeddings, output head, scalars/biases still need AdamW.
- vanilla Muon lets some weights grow too large over long training; needs added weight decay to fix (Moonshot).
- needs per-parameter update-scale (RMS) matching for stable large-scale training.
- SFT advantage vanishes when pretrain and finetune optimizers differ (authors' own caveat).
- Newton-Schulz only approximates orthogonalization - singular values end up noisy, not exactly 1.

Paper: https://t.co/88Uz99PWI3
Write-up: https://t.co/JGsAYHANGs
Code: https://t.co/MEMqAWOQgO

---

Get live 1-on-1 help and mentorship in a weekly group call - write papers, train LLMs, build AI projects every week LIVE with us: start your 7-day free trial: https://t.co/6nocqbVceu

[lumina-davhide @Luminadavhide]

@MrCharlesky Guide

[Mr Charles (Remote Jobs) @MrCharlesky]

Hiring: Customer Service Representative

📍Remote
Salary: 300,000 Naira

Duties:
1. Handle inbound customer calls and inquiries
2. Provide information and resolve customer issues
2. Maintain accurate customer records
Utilize strong computer skills.

Apply via comments

[$MAD 😡 Surge @GbemisolaE19923]

STAY $MAD RICH guys

@madrichclub_

[lumina-davhide @Luminadavhide]

@madrichclub_ @kkashi_yt And I’ve got $MAD on me for months @madrichclub_

[lumina-davhide @Luminadavhide]

I’m glad you love it brother

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD.

And guess I’m already becoming $MAD too

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy hopefully I get justified for making a $MAD tattoo on my body !!

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[lumina-davhide @Luminadavhide]

I’m glad you love it brother

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD.

And guess I’m already becoming $MAD too

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy hopefully I get justified for making a $MAD tattoo on my body !!

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[lumina-davhide @Luminadavhide]

I’m glad you love it brother

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD.

And guess I’m already becoming $MAD too

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy hopefully I get justified for making a $MAD tattoo on my body !!

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[MAD Rich Dev @madrichclub_]

@Luminadavhide SEMI-PERMANENT IS STILL $MAD imma actually have to get a real one on me in the future soon 😡👀

[lumina-davhide @Luminadavhide]

@madrichclub_ @solana_devs Guess I’m madder !!

DUHHH

Forgot to drop this

@madrichclub_

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

https://t.co/WSYmA1pgjM

[lumina-davhide @Luminadavhide]

A semi-permanent $MAD tattoo.
Guess I’m officially getting $MAD -der.

@madrichclub_ have you ever seen conviction at this level?

If the community loves it, I might just make it permanent.

Let’s go get $MAD

One thing’s for sure, I’ll be carrying $MAD with me for life.

[lumina-davhide @Luminadavhide]

Smash $MAD chads

@madrichclub_

[lumina-davhide @Luminadavhide]

@madrichclub_ I’m glad you love it brother

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD.

And guess I’m already becoming $MAD too

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy hopefully I get justified for making a $MAD tattoo on my body !!

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[lumina-davhide @Luminadavhide]

@madrichclub_ I’m glad you love it brother

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD.

And guess I’m already becoming $MAD too

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy hopefully I get justified for making a $MAD tattoo on my body !!

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[MAD Rich Dev @madrichclub_]

@Luminadavhide SEMI-PERMANENT IS STILL $MAD imma actually have to get a real one on me in the future soon 😡👀

[lumina-davhide @Luminadavhide]

@_someoneRandomn @Dezero_com Guess we all love that 100x portfolio 😂😂

[🚫BLANK🚫 @_someoneRandomn]

El Risitas in 2030 explaining how he just joined @Dezero_com wait-list and ended up with 100x portfolio 😂😂😂
#Dezero

[lumina-davhide @Luminadavhide]

@madrichclub_
I know you didn’t ask for this, but I just thought I’d shoot my shot

This tattoo is going to be with me for the next few months, and honestly I got it because I genuinely believe in $MAD. And yeah maybe I’m getting $MAD already 😂😂..

A little encouragement would make the pain of getting a $MAD tattoo feel a lot more justified

Either way, I’ll still be carrying $MAD around with me every day, and I think that’s pretty cool.

My SOL addy
3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

[MAD Rich Dev @madrichclub_]

@Luminadavhide SEMI-PERMANENT IS STILL $MAD imma actually have to get a real one on me in the future soon 😡👀

[lumina-davhide @Luminadavhide]

-STAY $MAD 😡

😡 😡 😡 😡😡
😡 😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡😡😡😡 😡 😡
😡 😡 😡😡 😡 😡😡

@madrichclub_

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

Make me more $MAD

[lumina-davhide @Luminadavhide]

A semi-permanent $MAD tattoo.
Guess I’m officially getting $MAD -der.

@madrichclub_ have you ever seen conviction at this level?

If the community loves it, I might just make it permanent.

Let’s go get $MAD

One thing’s for sure, I’ll be carrying $MAD with me for life.

[MAD Rich Dev @madrichclub_]

-STAY $MAD 😡

😡 😡 😡 😡😡
😡 😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡😡😡😡 😡 😡
😡 😡 😡😡 😡 😡😡

Copy the text above + your wallet + tag @madrichclub_

First 100 = 5K $MAD ($10)

Go.

[lumina-davhide @Luminadavhide]

@madrichclub_ Thanks so much my man

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

My wallet incase you wanna make me more mad

[lumina-davhide @Luminadavhide]

Smash this solana:Fa7ZE9nCEYnrHsnoeHuhEExJpchtrBtKXnWe6CgHpump chads

[lumina-davhide @Luminadavhide]

A semi-permanent $MAD tattoo.
Guess I’m officially getting $MAD -der.

@madrichclub_ have you ever seen conviction at this level?

If the community loves it, I might just make it permanent.

Let’s go get $MAD

One thing’s for sure, I’ll be carrying $MAD with me for life.

[MAD Rich Dev @madrichclub_]

@Luminadavhide SEMI-PERMANENT IS STILL $MAD imma actually have to get a real one on me in the future soon 😡👀

[lumina-davhide @Luminadavhide]

-STAY $MAD 😡

😡 😡 😡 😡😡
😡 😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡 😡 😡 😡
😡 😡 😡 😡😡😡😡 😡 😡
😡 😡 😡😡 😡 😡😡

3J3ULRdKqftSaksRZAZdZFFEFEsSKNWKCpDkHMb4nqws

@madrichclub_

[lumina-davhide @Luminadavhide]

@madrichclub_ @lit_terrestrial @Iam__dino9 @Sulexy77 @followdv80 Check this out bruh !! @madrichclub_

[lumina-davhide @Luminadavhide]

A semi-permanent $MAD tattoo.
Guess I’m officially getting $MAD -der.

@madrichclub_ have you ever seen conviction at this level?

If the community loves it, I might just make it permanent.

Let’s go get $MAD

One thing’s for sure, I’ll be carrying $MAD with me for life.