andrew lipschultz

[Ishan Khare @ishanskhare]

Excited to share Context-Sharded Block Parallelism (CSBP) and Turbo-dLLM, our new optimized distributed training library for Diffusion LLMs!

Our distributed parallelism strategy unlocks significant training efficiency for diffusion LLMs, with speedup gains growing with context length 🚀

On 8x H100 GPUs, Turbo-dLLM accelerates DFlash2 speculative-decoder training by 2.48x at 512K and 7.59x at 1M context length.

Simply pip install turbo-dllm⚡️ or check out https://t.co/Mcme2NOA3Y.

Honored to work with @TarunSures41845, @PranshuChatur11, @hangoo_kang, @pshroff_ , @KumbongHermann, and advisor @Azaliamirh 🌟!

[andrew lipschultz @andrewlipschul1]

@KloresLuke Get a day job

[andrew lipschultz @andrewlipschul1]

@stellamichel22 No one understands the time jumps change my mind

[andrew lipschultz @andrewlipschul1]

@PeteEdelstein This is hilarious

[andrew lipschultz @andrewlipschul1]

@russ_rosenthal No way

[andrew lipschultz @andrewlipschul1]

@Jack17307331 Bad idea

[andrew lipschultz @andrewlipschul1]

@walkerdm79 @SinaiBrain @EricJNestler Thank you Deena! You are the best