catherine

probably stuck in a local minimum rn | ML @Amazon prev @Google @Snap
All writing reflects my own views and personal work.

[catherine @cat_eye_on]

@PatrickToulme @icanvardar ok u cooked with this

[Yifan Wu @yifannnwu]

Been thinking about this for a while, as tasks go to more and more turns and longer horizons, PPO is much more elegant for giving dense, per-turn reward. And here we go.

[catherine @cat_eye_on]

@Stone_Tao @physical_int congrats Pebble Tao!

[catherine @cat_eye_on]

@neprodian What have you been doing to squeeze perf out of a single thread?

[catherine @cat_eye_on]

@Stone_Tao @jsuarez 👀

[catherine @cat_eye_on]

@creet_z @m_sirovatka DUDE HAHAHA

[catherine @cat_eye_on]

@theyangward Not super familiar with MARL, but does it make sense to have one policy per agent and an “orchestrator” policy

[catherine @cat_eye_on]

@theyangward Why doesn’t it scale?

[catherine @cat_eye_on]

@sheriyuo when did we start distilling dLLMs? first i’m hearing of this

[catherine @cat_eye_on]

@chenwanch1 @AIatMeta Congrats!

[catherine @cat_eye_on]

@JustinLin610 worded my thoughts perfectly

[catherine @cat_eye_on]

@yacinelearning the way this is how i learned what a load bearing token is

[catherine @cat_eye_on]

@himanshustwts sounds kinda interesting no?

[catherine @cat_eye_on]

@fujikanaeda whos your art guy 👀 @creet_z

[catherine @cat_eye_on]

@bilaltwovec Truly, I'm so excited to read the paper

[catherine @cat_eye_on]

@shreybirmiwal doesn’t need to be a competition, both are pretty cool

[catherine @cat_eye_on]

@shatayumk sparse rewards

[catherine @cat_eye_on]

@charles_irl @modal was gonna say you all are super quick with it but I guess you probably had access to this in advance haha

[wh @nrehiew_]

This paper prompted me to do a review of NVFP4 pre-training, given that NVIDIA seems to be pushing support for it especially on Blackwells.

Much of the content will come from "Pretraining Large Language Models with NVFP4" and the Nemotron 3 Super paper 🧵

[wh @nrehiew_]

The main highlight is that NVIDIA did NVFP4 pretraining. Much of the recipe follows previous Nemotron work:
- Hadamard Transforms applied to weight gradient computation to reduce the impact of outliers.
- Some layers kept at higher precision. (Table from Nemotron 3 Super). Specifically, final layers tend to require more dynamic range and mantissa than FP4 provides.
- Stochastic rounding rather than deterministic rounding to prevent bias, specifically in the gradients.

To validate the recipe, they train smaller models up to 16T and show a mere ~0.4% relative train loss gap with the bf16 baseline.

See more: https://t.co/XwBLFEg7QV

(we discuss other stability issues in later sections)