Early-stop builds on prior work:
https://t.co/QytYLWCyaH https://t.co/z1FLx4e2U7
We connect early-stop to flow maps: the clean prediction approximates the remaining transport to the endpoint.
Early-stop decodes the latest clean prediction before denoising finishes.
On GSM8K, ELF-REG-B at 64 NFE matches its own 512-NFE numerical answer in 87.11% of cases, but the complete response in only 8.61%.
5/8
I wrote a technical blog, and we open source our code and checkpoints.
Blog: https://t.co/AKnlGnEtxN
Paper: https://t.co/TAx2yjaGWi
Code: https://t.co/VjGPaJIemT
8/8
ELF-REG-L improves over ELF-L baseline on math and code:
GSM8K: 51.85% -> 55.96% (64 NFE)
HumanEval: 19.66% -> 22.56% (128 NFE)
Both are pass@1 at matched network function evaluations (NFE).
4/8
The answer can settle before the response does.
Introducing ELF-REG: we extend ELF, a fully continuous diffusion language model, to math and code with REPA+REG, and study early-stop few-step generation without few-step training.
1/8
We adapt representation alignment and entanglement (REPA+REG) from image generation.
A frozen AR teacher aligns intermediate features and supplies a global target denoised jointly with the response. The teacher is training-only.
3/8
Building on ELF, we denoise continuous representations of the entire response, then decode all tokens in parallel. We train task-specific models for math and code.
ELF: https://t.co/OWNtkJ0iGB
2/8