Dennis Y Menn

Ph.D. @utexasece. Building real-time video generation on consumer GPUs!

[Dennis Y Menn @MennDennis]

We can reach 20 FPS for real-time video editing on a single 4090.
1.3B self-forcing model, 4-step denoising, fp16.
Here's a screen recording of it running live on 4090. Repo in the comments.

[Jeff Liang @LiangJeff95]

Please checkout https://t.co/lOMvkBMJ46 which is a local AI filmmaking studio with your coding agent (Claude Code). Check it out and have a try.

[Chenfeng_X @Chenfeng_X]

Excited that our paper StreamdiffusionV2 received the Best Research Paper Award at #MLSys26!

🚀Video generation is quickly moving from demos to production-facing workloads. It is no longer a turn-based pipeline but should be a streaming pipeline to interact with users.

📖Our project page: https://t.co/ItuO5zc6hT and paper: https://t.co/fmz2irYIm1

👂Come join the talk if you are interested in streaming video generation. Our talk will be at the Research Track Oral Presentation: Best Paper Session on Tue 8:45AM at #MLSys26 , I will talk about how we attacked the efficiency and quality challenges. Hope to see you there!

❤️Huge thanks to all authors! This work would not have been possible without the incredible effort from the entire team. Big shout out to Tianrui Feng, Zhi Li, @Andy_ShuoYang , @HaochengXiUCB, @lmxyy1999 , @lvminzhang , @xiuyu_l , Keting Yang, @ZiqiPeng, @songhan_mit , @magrawala, @KurtKeutzer , and @cumulo_autumn

[Yating Wu @YatingWu96]

What does a scientific figure make you wonder? 📊

When we read papers, figures often raise questions that the surrounding text helps answer.

But most VLM benchmarks focus on questions answerable from the figure alone.

We introduce MQUD: 1,250 inquisitive questions over 245 figures from 56 papers, annotated by original paper authors.

MQUD extends Questions Under Discussion (QUD) from text to multimodal scientific discourse.

Instead of asking only what is visible, MQUD asks what implicit scientific question a figure raises in context.

[Chenfeng_X @Chenfeng_X]

🤔The more I studied diffusion language models, the more I came to appreciate the simplicity of autoregressive (AR) language models. AR models are trained to agree with what they generate, and their serving stacks are built to preserve that structure. DLMs often do neither: they lack introspective consistency, and high TPF does not necessarily translate into high real-world TPS.

We propose Introspective Diffusion Language Model (I-DLM), which unifies introspection and generation in a single pass:
1. 🧑‍🎓I-DLM brings introspective consistency to DLMs with only 5B training tokens, achieving AR-thinking-level quality.

2. 🚀 I-DLM carefully trades compute for higher TPF while converting that advantage into real TPS under high-concurrency serving.

📖Website: https://t.co/826V7d49mA
⌨️Code: https://t.co/MPuy6rAXbq

[Chenfeng_X @Chenfeng_X]

Test-time scaling with evolutionary search can cost hundreds of times more tokens than a single-shot generation. 😨 We built a unified framework for customizable evolutionary inference, with plug-and-play operators for selection, fitness estimation, and recombination. Check here: https://t.co/kM4PXahB9W 😎

More importantly, it cuts cost by orchestrating heterogeneous models, delivering up to 10× higher throughput while saving serious $$$ 💰.

A user-friendly API will come soon! 💪

[Daydream @DaydreamLiveAI]

Tomorrow (April 9), we're hosting @MennDennis for a Scope workshop where he'll walk through #LIPAR, a framework he built for accelerating real-time video editing with diffusion models.

What makes it interesting is that it requires no finetuning or retraining at all, and still achieves around 2x speedup with 20% less memory on a single RTX 4090, hitting 20 FPS with a 1.3B self-forcing model.

[Dennis Y Menn @MennDennis]

@chenlin_meng @pika_labs Would love to chat about how the speed-up pipeline could fit into Pika Stream!

[Dennis Y Menn @MennDennis]

We can reach 20 FPS for real-time video editing on a single 4090.
1.3B self-forcing model, 4-step denoising, fp16.
Here's a screen recording of it running live on 4090. Repo in the comments.

[Dennis Y Menn @MennDennis]

We can reach 20 FPS for real-time video editing on a single 4090.
1.3B self-forcing model, 4-step denoising, fp16.
Here's a screen recording of it running live on 4090. Repo in the comments.

[Chenfeng_X @Chenfeng_X]

Make your speculators work from DAY 0! Great collaboration work with @XiaoxiaWShirley @_junxiong_wang @ben_athi @ZhongzhuZhou @tri_dao @percyliang and other @togethercompute colleagues!

[Dennis Y Menn @MennDennis]

@slop_swap Thank you

[Dennis Y Menn @MennDennis]

We can reach 20 FPS for real-time video editing on a single 4090.
1.3B self-forcing model, 4-step denoising, fp16.
Here's a screen recording of it running live on 4090. Repo in the comments.

[Dennis Y Menn @MennDennis]

@hthillman Hi @hthillman, I enabled the direct message. Sure, let's discuss!

[Dennis Y Menn @MennDennis]

@dianamarculescu @Chenfeng_X Thanks again for the help from @AlbertYoung0112,
@WangBokun7
, Xiwen Wei, Mustafa Munir,
@LiangJeff95
, and valuable advice from Prof. Radu Marculescu for the LIPAR paper!

[Dennis Y Menn @MennDennis]

Thanks to the guidance from Prof. @dianamarculescu and Prof. @Chenfeng_X

[Dennis Y Menn @MennDennis]

Clarification: 4-step denoising is counted as (3 denoising steps + 1 memory step)

[Dennis Y Menn @MennDennis]

LIPAR seamlessly bridges the gap between traditional video compression algorithms and modern generative pipelines. Please see the link below:
🌐 Webpage: https://t.co/XT8vlypOMx
📄 Paper: https://t.co/gdpldhgBeH
💻 Code: https://t.co/wAH5n9vg4O

[Dennis Y Menn @MennDennis]

Tweet 1
We achieve real-time video editing at 19.3 FPS on a single RTX 4090 (using just 16GBVRAM) 🚀

Introducing LIPAR: We exploit temporal redundancy in latent space to boost Self-forcing throughput by 1.53× with ZERO additional training with no drop in performance!
@_akhaliq

[Dennis Y Menn @MennDennis]

@w11123321 Thank you

[Dennis Y Menn @MennDennis]

Huge thanks to the guidance from my advisor, Diana Marculescu, as well as my incredible co-authors Yuedong Yang, @WangBokun7, Xiwen Wei, Mustafa Munir, @LiangJeff95, and Radu Marculescu and @Chenfeng_X.