Ishan Khare

ML Research @StanfordAILab, @hazyresearch

[Jacky Kwok @jackyk02]

Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions.

CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks.

With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks.

We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed.

Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size.

📄 Blog: https://t.co/zwi9JOHKGx
💻 Code: https://t.co/rsHRYCGR8I
🗣️ Discord: https://t.co/Uqtdefvo3J
🤗 Data & Models: https://t.co/wdSWGGO3hu

More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵

[Ishan Khare @ishanskhare]

@kirillacharya exciting stuff!

[Ishan Khare @ishanskhare]

@FrancoisChauba1 fable IS reviewer #2 for conferences these days

[Ishan Khare @ishanskhare]

Excited to share Context-Sharded Block Parallelism (CSBP) and Turbo-dLLM, our new optimized distributed training library for Diffusion LLMs!

Our distributed parallelism strategy unlocks significant training efficiency for diffusion LLMs, with speedup gains growing with context length 🚀

On 8x H100 GPUs, Turbo-dLLM accelerates DFlash2 speculative-decoder training by 2.48x at 512K and 7.59x at 1M context length.

Simply pip install turbo-dllm⚡️ or check out https://t.co/Mcme2NOA3Y.

Honored to work with @TarunSures41845, @PranshuChatur11, @hangoo_kang, @pshroff_ , @KumbongHermann, and advisor @Azaliamirh 🌟!

[Tarun Suresh @ COLM 2026 @TarunSures41845]

Diffusion LLMs and speculative decoding promise much faster agents. Yet agents need long contexts, and training on them is painfully slow.

Introducing Context-Sharded Block Parallelism (CSBP), a new distributed parallelism strategy unlocking significant training efficiency for diffusion LLMs, with speedup gains growing with context length 🚀

⚡ 7.59× faster DFlash2 speculative decoding drafter training
⚡ 1.61× faster block diffusion fine-tuning
⚡ 1.33× faster autoregressive → block diffusion adaptation

With the same GPU hours, models trained with CSBP score higher on SWE-bench Verified and Terminal-Bench Lite 👑

Open-sourced in Turbo-dLLM, our new optimized distributed training library.

Advised by @Azaliamirh and with an amazing team: @PranshuChatur11 @hangoo_kang @pshroff_ @ishanskhare @KumbongHermann

[Ishan Khare @ishanskhare]

@alexshander03 devin like me fr can we fix him? @devmchheda

[Ishan Khare @ishanskhare]

@thewilliampan bro got hella likes guess post got boosted internally 😂

[Ishan Khare @ishanskhare]

@distributionat @saarth_ always an exception to the rule

[Ishan Khare @ishanskhare]

@brianryhuang

[Ishan Khare @ishanskhare]

@FrancoisChauba1 2.2B

[Ishan Khare @ishanskhare]

@Anqinator @FrancoisChauba1 whoa crazy crossover event 🤯

[Ishan Khare @ishanskhare]

@jeremytian_ my linkedin king has returned to X

[Ishan Khare @ishanskhare]

@lucyjcai @tobefranklee we from the same town in ohio 😂

[Ishan Khare @ishanskhare]

@tobefranklee @lucyjcai whoa when my worlds collide 😮

[Stuart Sul @stuart_sul]

(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs.

Joint work with @nash_c_brown, @hmwildermuth, @tmwilliamlin168, and @ellev3n11

[Ishan Khare @ishanskhare]

@stuart_sul Very exciting work!

[Jerry Liu @jerrywliu]

MLPs store facts in language models. Can we write them into Transformers without training?

New work w/ amazing team @garctrob @ronnygjunkins @EyubogluSabri, Atri Rudra & @HazyResearch gives a ✨closed-form✨ recipe for fact-storing, Transformer-ready MLPs. Accepted at COLM 2026!

[Ishan Khare @ishanskhare]

score predictions:

🏴󠁧󠁢󠁥󠁮󠁧󠁿 England 2-0 Dr Congo 🇨🇩

🇧🇪 Belgium 1-2 Senegal 🇸🇳

🇺🇸 USA 3-1 Bosnia and Herzegovina 🇧🇦

[Aneesh Muppidi @aneeshers]

Introducing Real-time RL.

In the real world, time isn't free. The environment keeps "moving" even when you're computing your next action.

We show how RL agents can learn to adaptively think in real-time games.

1/🧵

[Ishan Khare @ishanskhare]

@Willob the original for those who don't know: https://t.co/NPmLIMNvBG

[Ishan Khare @ishanskhare]

@probablynotaz9 the 🐐 of research