Seil

Visiting Scholar, @StanfordAILab, @ScalingIntelLab
PhD Student, CS @yonsei_u

[Seil @seilk_]

Introducing ThunderSyncRL ⚡ It makes agentic RL up to 𝟭.𝟵× 𝗳𝗮𝘀𝘁𝗲𝗿 with 𝘇𝗲𝗿𝗼 𝗽𝗼𝗹𝗶𝗰𝘆 𝘀𝘁𝗮𝗹𝗲𝗻𝗲𝘀𝘀.

We observed that synchronous GRPO/OPD leaves learner GPUs idle for long stretches (𝟰𝟵.𝟱% of step time when training on SWE-smith). Asynchronous training avoids this, but it learns from 𝘀𝘁𝗮𝗹𝗲 𝗽𝗼𝗹𝗶𝗰𝗶𝗲𝘀 and performs worse. Our goal was to keep GPUs maximally utilized while still doing strictly on-policy updates.

ThunderSyncRL streams gradients as soon as their inputs are ready, then takes one synchronous optimizer step. We prove the GRPO and OPD updates are identical to batch-synchronous training.

On SWE-bench Verified and Terminal Bench 4.0, it reaches Sync's pass@3 up to 𝟭.𝟵× 𝗳𝗮𝘀𝘁𝗲𝗿 and beats Async by up to 𝟮.𝟰𝟳 𝗽𝗼𝗶𝗻𝘁𝘀 of pass@3 at the same GPU budget.

Grateful to my co-lead @hangoo_kang and to the amazing @Azaliamirh, who made this work possible.

And a big thank you to @TarunSures41845, Youngeun Kim, @sayshrey, and Seong Jae Hwang for building it with us. (1/7) 🧵

📄 Paper: https://t.co/T7fOLN2nLo
👾 Code: https://t.co/ky5Py3tfeh
🌐 Blog: https://t.co/Y3cKPzvMpZ

[Ishan Khare @ishanskhare]

Excited to share our work at the discrete diffusion reading group on Monday!

Feel free to reach out with questions or ideas — always happy to chat!

paper: https://t.co/aSKSxueiBl
github: https://t.co/wLT00bGK0t

[Shayan Talaei @talaei_shayan]

Introducing Speculative Self-Distillation (SSD, NeurIPS 2026), which pushes the accuracy–efficiency–forgetting frontier of self-distillation.

In SSD, student and teacher (the same model with additional information) write the rollouts together. Like a private tutor, the teacher lets the student write and takes over only when the student drifts too far. As a result, SSD trains on rollouts that stay as close to the student's own writing (on-policy) as possible while the teacher keeps them on track.

📉 38–69% fewer training tokens than on-policy, matching its accuracy
⏱️ Reaches on-policy accuracy in ~130 vs ~414 min (Science Q&A)
🧠 Forgets less in a stress test: −1.4 vs −4.5 points for on-policy

[Seil @seilk_]

I really like work like this! SSD lets the student write and the teacher step in only to fix the facts. An obviously efficient way for self-distillation. SSD teaches LLMs new knowledge in up to 69% less time (!)

Really cool work, @talaei_shayan!

[Shayan Talaei @talaei_shayan]

Introducing Speculative Self-Distillation (SSD, NeurIPS 2026), which pushes the accuracy–efficiency–forgetting frontier of self-distillation.

In SSD, student and teacher (the same model with additional information) write the rollouts together. Like a private tutor, the teacher lets the student write and takes over only when the student drifts too far. As a result, SSD trains on rollouts that stay as close to the student's own writing (on-policy) as possible while the teacher keeps them on track.

📉 38–69% fewer training tokens than on-policy, matching its accuracy
⏱️ Reaches on-policy accuracy in ~130 vs ~414 min (Science Q&A)
🧠 Forgets less in a stress test: −1.4 vs −4.5 points for on-policy

[Azalia Mirhoseini @Azaliamirh]

Had a great time at AGI House last Sunday talking about Ripple-65B Preview and how to optimize the cost of intelligence.

https://t.co/3fepy1KjZX

[Azalia Mirhoseini @Azaliamirh]

Introducing Joules-to-GDP: a new metric and framework for measuring how much economic value AI generates per unit of energy.

Key insight: selecting the right model on the right accelerators can deliver several times more value per joule for the same task.

We believe as AI serving scales across every part of the economy, optimizing for this metric will matter increasingly more.

Great work co-led by @amirzeinaalii @JonSaadFaclon and @Avanika15, with an amazing team @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, and @HazyResearch.

Check out Jon's thread for several other interesting findings:

[Shayan Talaei @talaei_shayan]

I made a 3D visualization of Ripple-65B Preview's architecture! Check it out here (with an easter egg): https://t.co/zadU1omLPS

Ripple is a dense model with 80 layers and a width of 5,120, so its aspect ratio is 64. That makes it very deep for its width. In a model this deep, each layer keeps adding its output to the residual stream, and by the deeper layers the original token information gets diluted.

Most recent architectures handle this by changing the residual stream itself, like DeepSeek's manifold-constrained hyper-connections (mHC) or Kimi's Attention Residuals. We went with value residuals instead. Every layer mixes the original token embedding back into its attention values, and a learned weight in each layer decides how much.

[Jon Saad-Falcon @JonSaadFalcon]

AI is using more and more of the world’s energy. But how much economic value are we getting from AI?

We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.

Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:

1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.

2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.

3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.

With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.

Joint work co-led by @amirzeinaalii and @Avanika15, with @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, @HazyResearch, and @Azaliamirh.

More details below 🧵

[Seil @seilk_]

Congrats on the great work, Amir!!

[Amir Zeinali @amirzeinaalii]

AI is running into energy limits, and people are asking whether AI investments will earn back what they cost. That’s exactly why this moment is so exciting. We need to focus on the sectors and workloads that create the most value, and build models and chips that do more with every unit of energy.

Really enjoyed working with @JonSaadFalcon, @Avanika15, and the entire team on Joules-to-GDP! Thanks to everyone who helped along the way, especially @Jon_Hartley_, @Azaliamirh, and @HazyResearch.

[Seil @seilk_]

Such a timely new evaluation on a dimension that hasn't been explored in this much detail before! Crucial findings on the trade-offs between model size, hardware scaling, and actual economic return per joule!

[Jon Saad-Falcon @JonSaadFalcon]

AI is using more and more of the world’s energy. But how much economic value are we getting from AI?

We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.

Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:

1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.

2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.

3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.

With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.

Joint work co-led by @amirzeinaalii and @Avanika15, with @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, @HazyResearch, and @Azaliamirh.

More details below 🧵

[Seil @seilk_]

Really cool work @talaei_shayan! Congrats on the huge milestone!!

[Shayan Talaei @talaei_shayan]

Really happy to finally share Ripple-65B Preview! We trained it from scratch with a core team of five researchers, and it sets a new U.S. open-weight SOTA on Terminal-Bench 2.1 for agentic coding.

Three choices that shaped how we built it, and a few things I learned along the way.

1. We deliberately chose a dense model.

Many leading open-weight models are MoEs. For our use case, we wanted strong reasoning and low latency for agents going through long sequences of model and tool calls. Going dense also gave us a simpler training and serving stack and avoided expert-routing communication. That combination mattered a lot to us.

2. We bought no posttraining data.

A lot of the work went into building our own data generation and filtering pipelines, and making sure we covered the tasks we cared about. We built on open-source training and inference stacks throughout. I’m particularly happy about how much we were able to do as a small team by building on the community’s work.

3. We worked on the RL system and curriculum together.

Mixed RL meant dealing with very different rollout lengths, tool-call patterns and shares of the batch across domains. We tuned oversampling per domain and per problem to keep batch latency down, while managing policy staleness and trainer-inference mismatch. Score centering worked best among the corrections we tried; we used it alongside truncated importance sampling.

We also kept adjusting task selection and introducing harder problems as the model improved. Getting batches ready faster and keeping updates stable had to go hand in hand with giving the model useful problems to learn from.

Loved building this with @DZhang50, @esonghori2, @ethanboneh and @amiwilliamhuang. Huge thanks to our cofounders @annadgoldie and @Azaliamirh, and the whole @RicursiveAI family for making it possible. Really proud of what we built together.

[Seil @seilk_]

Huge congrats to @Azaliamirh and the team! Super impressive results!!

[Azalia Mirhoseini @Azaliamirh]

Very excited to launch Ripple-65B Preview, a new state of the art for U.S. open-weight model on agentic coding!

You can run Ripple-65B locally and efficiently on a MacBook Pro or DGX Spark!

Give it a try:
HuggingFace: https://t.co/o0MmMnpi6N
Together AI: https://t.co/ecz2drgKff
GMI Cloud: https://t.co/ajjgAKXRlD
OpenRouter: https://t.co/ltemSvuVg4

[Shayan Talaei @talaei_shayan]

Really happy to finally share Ripple-65B Preview! We trained it from scratch with a core team of five researchers, and it sets a new U.S. open-weight SOTA on Terminal-Bench 2.1 for agentic coding.

Three choices that shaped how we built it, and a few things I learned along the way.

1. We deliberately chose a dense model.

Many leading open-weight models are MoEs. For our use case, we wanted strong reasoning and low latency for agents going through long sequences of model and tool calls. Going dense also gave us a simpler training and serving stack and avoided expert-routing communication. That combination mattered a lot to us.

2. We bought no posttraining data.

A lot of the work went into building our own data generation and filtering pipelines, and making sure we covered the tasks we cared about. We built on open-source training and inference stacks throughout. I’m particularly happy about how much we were able to do as a small team by building on the community’s work.

3. We worked on the RL system and curriculum together.

Mixed RL meant dealing with very different rollout lengths, tool-call patterns and shares of the batch across domains. We tuned oversampling per domain and per problem to keep batch latency down, while managing policy staleness and trainer-inference mismatch. Score centering worked best among the corrections we tried; we used it alongside truncated importance sampling.

We also kept adjusting task selection and introducing harder problems as the model improved. Getting batches ready faster and keeping updates stable had to go hand in hand with giving the model useful problems to learn from.

Loved building this with @DZhang50, @esonghori2, @ethanboneh and @amiwilliamhuang. Huge thanks to our cofounders @annadgoldie and @Azaliamirh, and the whole @RicursiveAI family for making it possible. Really proud of what we built together.

[Seil @seilk_]

16x pretraining compute efficiency and 70x fewer RL rollouts to outperform Muse and hit SOTA at 1/10th the cost!! Their end-to-end pretraining and recipe is seriously impressive!!

[Ricursive Intelligence @RicursiveAI]

Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.

Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.

Learn more: https://t.co/Pe04ZKhkE9

[Rikiya Takehi @rikiyatakehi]

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq

[Tarun Suresh @ COLM 2026 @TarunSures41845]

Exicitng work from @shreyask269 and @madhav_kanda_ on accelerating block diffusion spec decoding!

[Azalia Mirhoseini @Azaliamirh]

We’re releasing our Blackwell kernel verifier, building on our recently released work on "AI as a Compiler." In this approach, an LLM generates optimized PTX for a given workload, and our verifier checks it for correctness. By skipping intermediate compiler layers, we enable finer-grained control over the hardware at the PTX level, paving the way for more efficient kernels and models.

The verifier checks several properties, including symbolic correctness, race conditions, and synchronization issues.

You can get access here:

Volta Blackwell Verifier: https://t.co/ufcqaM8vli
Triton AI Compiler: https://t.co/VK8DRBY24G

And here is the paper:
AI as a Compiler: https://t.co/B2wpy5VwuT

[François Costa @cos_francois]

Motivated by growing industry interest, we're excited to share our initial efforts toward stronger correctness guarantees for AI compilation and GPU kernel generation!

Our verifier, Volta Blackwell, analyzes NVIDIA Blackwell ML kernels by extracting a happens-before relationship graph alongside symbolic expressions representing their computations. By performing correctness checks over these representations, it can detect race conditions, synchronization issues, and incorrect expressions that conventional kernel testing might miss.

Using this approach, Volta Blackwell identified subtle fence-related issues in our Triton AI compilation pipeline that would not have been caught by the standard correctness tests proposed by KernelBench. This highlights the limitations of test-based validation and the importance of formal reasoning about GPU kernel execution.

Developing comprehensive and sound verifiers is a critical next step for AI-driven kernel generation. Subtle synchronization, memory-ordering, and computational correctness bugs remain major challenges, and no existing verifier yet provides the end-to-end guarantees required for fully trusted AI compilation and GPU kernel generation for ML workloads.

💻 Volta Blackwell: https://t.co/7fgF3iz98t
💻 Triton AI Compiler: https://t.co/owvOWgJIFL
📄 AI as a Compiler: https://t.co/D9ow1YMvL2

[Yuzhen Mao @Mao_Yuzhen]

Was just as intrigued as everyone else to read Anthropic’s new post on folding subagents back into the main agent loop.

We ran into the same issues while working on DeLM, but came away with a slightly different conclusion. Having spent quite a bit of time studying those trajectories, I thought I’d share what we found 👇.

1/4

[Tarun Suresh @ COLM 2026 @TarunSures41845]

Diffusion LLMs and speculative decoding promise much faster agents. Yet agents need long contexts, and training on them is painfully slow.

Introducing Context-Sharded Block Parallelism (CSBP), a new distributed parallelism strategy unlocking significant training efficiency for diffusion LLMs, with speedup gains growing with context length 🚀

⚡ 7.59× faster DFlash2 speculative decoding drafter training
⚡ 1.61× faster block diffusion fine-tuning
⚡ 1.33× faster autoregressive → block diffusion adaptation

With the same GPU hours, models trained with CSBP score higher on SWE-bench Verified and Terminal-Bench Lite 👑

Open-sourced in Turbo-dLLM, our new optimized distributed training library.

Advised by @Azaliamirh and with an amazing team: @PranshuChatur11 @hangoo_kang @pshroff_ @ishanskhare @KumbongHermann

[arXiv Bangers @arXivBangers]

ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learning

ThunderSyncRL computes gradients as inputs arrive, matching synchronous mathematical updates while training agentic models up to 1.9x faster without policy staleness.

https://t.co/JammVD0Zgz