Andrei Filatov

doing gucci diffusion stuff

[Thomas Fel @thomas_fel_]

A science of post-training can be built on interpretability.

Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules.

In vision, DPO & RL, and its effects on monitoring 🧵

[Andrei Filatov @anvilapex]

gucci stuff

[Krea @krea_ai]

the Krea 3 technical report is out.

a deep dive on the data, architecture, and training techniques used to create Krea 3.

learn more 👇

[Andrei Filatov @anvilapex]

@rgvrmdya yeah, human expert judgments are still the gold standard. But at scale, with multiple RL rounds, it's hard to beat model-based judges as you can't produce so many human annotations. humans work best for calibrating models

[Andrei Filatov @anvilapex]

wrote up part 2 of RL for diffusion: reward models + loss balancing

https://t.co/uHbM8mW1bw

its quite fascinating how intuitive RL algorithms are — and how hard it is to design a proper reward.

[Andrei Filatov @anvilapex]

4/ result: tuning ~2k numbers beats LoRA-GRPO with 26M. GSM8K 0.867 vs 0.823 (full-weight GRPO: 0.870), IFEval 0.555 vs 0.483

and up to 7.8x faster to the same MATH500 accuracy on Qwen2.5-32B, 128 GPUs

https://t.co/bRfPibX5W6

[Andrei Filatov @anvilapex]

still kind of amazed that RL for diffusion is basically smart SFT: SFT decides where the weights move, the reward decides how much

turns out LLMs have the same thing, ESSA (https://t.co/bRfPibX5W6) splits it exactly this way

[Andrei Filatov @anvilapex]

3/ how to do that RL? evolution strategies are back and fit here: a 7B model has only ~2k singular values

CMA-ES (Covariance Matrix Adaptation Evolution Strategy, https://t.co/KhPo5RkOvh) samples candidates from a Gaussian and moves it toward the best ones. no backprop

[Andrei Filatov @anvilapex]

@sleenyre looks nice

[Andrei Filatov @anvilapex]

wrote up some thoughts on rl for diffusion
https://t.co/jaYtRUwaxr

at the end of the day, it’s just weighted flow matching.

[Andrei Filatov @anvilapex]

@NagaSaiAbhinay come to dubai, habibi - best place

[Andrei Filatov @anvilapex]

@flappyairplanes ✈️

[Andrei Filatov @anvilapex]

@ErwannMillon erwann-boss 😎

[Andrei Filatov @anvilapex]

@krea_ai 🚀🚀🚀🚀

[Andrei Filatov @anvilapex]

Wrote up how diffusion DPO learns preferences, and why a lower loss can still mean worse images:

https://t.co/jDx7S3KkHH

[Andrei Filatov @anvilapex]

@cloneofsimo i'm waiting for ai to start proving theorems. the latest progress is usually based on founding some examples/counterexamples to problems - its major progress but setting up the new field throught new theorems i think it requires something smarter ai than we have right now

[RadixArk @radixark]

Miles natively supports @sgl_project for fast rollouts, while keeping rollout and training aligned for reliable post-training at scale.

Grateful to the community for the contributions and feedback that continue to shape Miles.

[Andrei Filatov @anvilapex]

@agermanidis Congrats

[Andrei Filatov @anvilapex]

@krea_ai @TitusTeatus Titus boss

[Andrei Filatov @anvilapex]

make this fight in mountains