Excited to announce Next-Concept Modeling (NCM): our new paradigm for fast and parallel, coarse-to-fine text generation! 🚀
Compared to strong diffusion language models (DLMs) at similar parameter sizes, NCM showcases:
⚡ 31× faster generation on downstream summarization and translation tasks.
🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity using 3–5× fewer training tokens.
🧩 Better reasoning performance on tasks like Sudoku.
Check out the paper and codebase 👇
Arxiv: https://t.co/NUCOYycgcM
AlphaXiv: https://t.co/4brvP8GxDg
Code: https://t.co/CnkJHS4KQt
Enjoyed working with @KumbongHermann and advised by @simran_s_arora@Azaliamirh and @HazyResearch.
I'm starting a podcast, "Confession," where I invite tech veterans to share their painful lessons.
We all connect through shared pain. Hear founders & CEOs like Satya Nadella (Microsoft), Dara Khosrowshahi (Uber), Luana Lopes Lara (Kalshi), and others share their stories and lessons learned.
Subscribe to Confession on...
✅ Spotify: https://t.co/ivOCa9wYK5
✅ Youtube: https://t.co/Ofo7boNWfg
✅ Apple: https://t.co/i8SFYF1CZC
@SatyaNadella@Dkhos@luanalopeslara
Excited to share Context-Sharded Block Parallelism (CSBP) and Turbo-dLLM, our new optimized distributed training library for Diffusion LLMs!
Our distributed parallelism strategy unlocks significant training efficiency for diffusion LLMs, with speedup gains growing with context length 🚀
On 8x H100 GPUs, Turbo-dLLM accelerates DFlash2 speculative-decoder training by 2.48x at 512K and 7.59x at 1M context length.
Simply pip install turbo-dllm⚡️ or check out https://t.co/Mcme2NOA3Y.
Honored to work with @TarunSures41845, @PranshuChatur11, @hangoo_kang, @pshroff_ , @KumbongHermann, and advisor @Azaliamirh 🌟!
Biological engineering has long aimed to design living systems with the programmability of software, but the tools to do so rarely fit together.
Today, we’re sharing our preprint introducing Proto, a high-level programming language for generative biology that composes models and design tools across DNA, RNA, and proteins to optimize towards user-specified functional objectives.
It’s finally here: real-time voice hallucination correction.
Exciting to turn a race against the audio playhead into something reliable on live calls.
Check out the article for more https://t.co/LuIBWA8YA0