Flow Matching connects two distributions.
Can a one-step generator learn from the flow between its own samples and real data?
Meet Midpoint Generative Models (MGM), our new training framework built on this idea.
Here’s how. 🧵
We have done a major update on distillation and speculative decoding, and release our Nemotron-Labs-Sigma model family on Huggingface. Check it out at https://t.co/VtUEfBxyf9
Kudos to all collaborators and looking forward to the future of continuous diffusion LLMs!
Big reasoning speedups are here. 1,000× faster training and 280× less compute than any looped model. All done using a fully trainable energy from the 80s
Keep reading to hear how energy is the right paradigm for reasoning
With @HughBlayney@jcrwhittington@prefrontal_ai
🧵/10
Can diffusion models be used for prime factorization? In our new paper with @bodonoghue85, @weballergy, @demishassabis, and others, we explore the limits of this approach.
For small targets, transformer-based discrete diffusion combined with a novel beam search procedure can reliably predict the factor bits of held-out semiprimes. We pushed the ML-based SOTA from 56 to 68 bits, and conducted an empirical scaling study across data, compute, and problem size.
The key bottleneck: problem size. Unlike natural language, which exhibits easily exploitable local substructure, prime factorization lacks the kind of smooth inductive bias neural nets thrive on - and our unfavorable scaling curves suggest a different approach is needed.
We’re excited to share these findings to provide a rigorous baseline and help guide the community on where to look next.
One stale cached system prompt raised the trainer-sampler KL 80x in TRL's AsyncGRPO on vLLM: 0.082 vs 0.001 with a per-version cache salt, steps 151-200.
The lesson for async RL: any cache that outlives a weight sync must be reset or keyed by weight version.
Our survey on continuous diffusion and flow language models is out! ❤️
A little animated tour of what’s inside 👇 From representations and training to few-step generation, scaling, and open problems.
https://t.co/BJOVac0Bz9
New survey on Continuous Diffusion and Flow Language Models is out! 🌊💬
One of my first papers, back in early 2025, was on continuous simplex flows. After working on discrete diffusion, I wanted to try controllable sequence generation with flow matching, which naturally operates in continuous spaces. But at the time, I found them much harder to train well than discrete diffusion. 🥲
2026 has changed this picture completely! 🚀 It’s been super exciting to watch continuous LMs take off, but with the rapid advancements, it’s also becoming harder to keep up with all the new work across pre-training, few-step generation, guidance, and scaling 🥹
So together with @chandavidlee, @FEijkelboom, and @jw9730, we put together a survey that consolidates the continuous LM literature and its key insights into 37 pages. It covers everything from the early work before the so-called “winter” of continuous LMs to the wave of new papers in 2026 🌟
More in thread! 🧵
Latent diffusion for text usually freezes the encoder before diffusion training, so the diffusion model gets no say in its own latent space. We let its gradient in.
LDLM, NeurIPS 2026. Code and checkpoints below.
At NeurIPS in Paris? Let's grab coffee and talk about latent diffusion and latent communication between models.
Excited to share that our paper, “Kernel-Gradient Drifting Models,” has been accepted to NeurIPS 2026! ❤️
📄 Paper: https://t.co/WeDMRyrqbM
💻 Code: https://t.co/7ugk1cwaFf
wrote up part 2 of RL for diffusion: reward models + loss balancing
https://t.co/uHbM8mW1bw
its quite fascinating how intuitive RL algorithms are — and how hard it is to design a proper reward.
Happy to share our most recent work:
An ODE-based Sampler for Discrete Diffusion Models
- we introduce a probability-flow analogue for discrete diffusion: deterministic dynamics that preserve its time marginals.
- we derive a principled discretization and efficient implementation for both MDM and UDM.
- this allows a theoretically grounded consistency distillation objective.
https://t.co/lOMB1zQL5i
Joint work with @samsongvch, @ericxing, @EricMoulines, @AlainDurmus and Giacomo Zanella
We are very excited about this direction and believe that it can be pushed much further!
(1/n)
i trained a 14M-parameter world model on 2D Minecraft in under 10 minutes for $0.1 and turned it into an autoresearch benchmark.
no game engine. the network draws every pixel. play it locally on your phone.
https://t.co/kPFQKQFQ9Q
Introducing JetSpec++: we find causal parallel drafting paired with hidden states predictor and anchor token decoder can effectively draft the next future block before the current one is verified, and spending the freed time on self refinement.
This allows drafting to be pipelined with verification, hiding almost all of its drafting cost.
JetSpec++ reaches up to 1.54x end-to-end speedup over DSpark and 7.45x over autoregressive decoding while keeping lossless, at 1,361 TPS on Qwen3-8B and 367 TPS on Qwen3.8-27B in BF16.
With NVFP4 and kernel optimizations, JetSpec++ further reaches over 4,000 TPS on Qwen3-8B and over 900 TPS on Qwen3.8-27B, on two GB200 GPUs. ⚡️
Project page: https://t.co/tuhf2KPfsW
Neural Thickets (@yule_gan@phillip_isola): add random noise to an LLM's weights, keep the best copies, let them vote. Reasoning jumps, no training.
Why does it work? It's sampling in disguise. Swap noise on billions of weights for one random vector on the logits: same gains 🧵
Energy-Based Models can now scale better than Diffusion models.
Introducing TEM: Transformer Energy-based Models
TLDR:
��� TEM outscales Diffusion without guidance
• First scalable contrastive Energy-Based Models
• Better generalization, test-time scaling, and OOD detection than Diffusion
🧵
What makes a good representation for diffusion models?
Surprisingly, more information isn't always better.
In our new work, Purify Before You Align, we show that removing locally predictable information can substantially improve diffusion alignment.
🎥 A quick overview 👇
🧵 1/n
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 https://t.co/Cd2qiugCJo
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
Introducing Locals: an Apple Silicon engine that runs local LLMs up to 3x faster.
It is built with a single mission: to free you from cloud providers and run frontier-class models locally without needing massive compute resources.
It’s simple. Run our engine, and your coding harnesses and autonomous agents can immediately use it as a drop-in replacement for Ollama, MLX-LM, or llama.cpp.
Under the hood is a theoretical breakthrough in complexity theory, enabling substantially faster algorithms for DeltaNet architectures, grammar-constrained decoding, tool calling, and structured outputs.
Open source and available now on GitHub.
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨
Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)