Krish Agarwal

ECE PhD student @CarnegieMellon

[Krish Agarwal @wonderingkrish]

Happy to share our recent work on FlashRT ⚡

Multimodal applications are highly diverse. Most systems work still falls on human developers, which is unscalable.

FlashRT automates this by guiding coding agents to write efficient serving infrastructure with no human intervention!

[Infini-AI-Lab @InfiniAILab]

AI is now building the infrastructure it runs on.

Getting real-time multimodal applications — including live avatars, voice agents, video world models — to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70× lower latency and 3.6× higher throughput, across 5 real-time apps.

Today we release FlashRT ⚡, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention.

✅ Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc.
✅ Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware you’re on
✅ Can easily compose with third-party serving engines, the agent automatically implements orchestration

📄 Paper: https://t.co/TMaeHRIpXM
🌐 Website: https://t.co/sikq8aYwC5
🔗 GitHub: https://t.co/4NPKZi3SpQ

🧵1/5

[Beidi Chen @BeidiChen]

📢AI is taking over the infrastructure it runs on—and already doing better. FlashRT achieves up to 70× lower latency and 3.6× higher throughput across five real-time apps, even delivering stronger results on less human-optimized AMD GPUs than on NVIDIA.

We used to handcraft static systems, but human expertise doesn’t scale. In the agent era, every application should have an end-to-end system customized by AI for its workload and hardware.

FlashRT is an early step toward that future.

[Beidi Chen @BeidiChen]

📢Releasing MonarchRT 🦋, an efficient architecture for real-time, interactive video generation. Butterfly structures and Monarch have been favorites in our early LLM explorations for linear layers @tri_dao (heard @OpenAI has been using it in 5.2 lately); video attention is their new home due to the intrinsic structure.

Big potential for world models in gaming (Genie3-style) and robotics (DreamZero–style @DrJimFan). As systems get interactive, latency matters more than raw FPS —and real efficiency gains on top of few-step AR diffusion without quality loss are still very hard.

Check out MonarchRT for how we leverage video structure to make this possible. Congrats @InfiniAILab on new work in a new direction on Day 1 of Lunar New Year 🧧 💪

[Krish Agarwal @wonderingkrish]

Super excited to share the first work of my PhD! MonarchRT🦋 enables true real-time video generation on a single RTX 5090🚀

This is one of the first sparse attention parameterizations to actually work for real-time video models.

Happy Lunar New Year🧧 and Happy Mahashivratri 🔱

[Infini-AI-Lab @InfiniAILab]

Video generation models are improving fast—real-time autoregressive models now deliver high quality at low latency, and they’re quickly being adopted for world models and robotics applications. So what’s the problem? They’re still too slow on consumer hardware.

🚀 What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200)

Today we release MonarchRT 🦋, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains.

📄 Paper: https://t.co/d1AAMIseow
🌐 Website: https://t.co/41mqriKekx
🔗 GitHub: https://t.co/hp5iJttviA

🧵1/n

[PixVerse @PixVerse]

Finite space, infinite stories.
"Aquarium" by @kikkawa_mese shows the power of PixVerse R1.
Each generation breathes new life into the tank, creating a continuous, interactive loop of digital evolution.
The era of "Infinite Flow" AI video is here.
Rt+Reply+Follow = Invite Code (50 Random Pick)

[Beidi Chen @BeidiChen]

The whole @InfiniAILab is at #NeurIPS this week! Our group is currently working on diverse directions of GenAI, .e.g., Scalable and Efficient RL, VideoGen, Modeling, Model Arch & Sys Co-Design (Many new releases coming!!). Come and talk to us @RJ_Sadhukhan @IronSteveZhou @chenzhuoming911 @Xinyu2ML @haizhong_zheng @wonderingkrish @LiuAtlas89429

[Infini-AI-Lab @InfiniAILab]

🥳 Happy to share our new work –  Kinetics: Rethinking Test-Time Scaling Laws

🤔How to effectively build a powerful reasoning agent?

Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model.
But, It only shows half of the picture!

🚨 The O(N²) KV memory access in self-attention dominates the cost of test-time scaling (TTS).
MoEs even worsen memory bottleneck by cutting compute.

Our new scaling law Kinetics suggests investing in model size first before spending more in test-time compute.

This insight leads to our next key finding
✨ Sparse Attention = Scalable TTS

Our Kinetics sparse scaling law says that when doubling the resources, we should prioritize increasing test time tokens over attention density.

✅ 60+ pts improvement under the same compute budget
✅ 10× lower resource usage for equivalent performance
✅ Sparse attention becomes increasingly valuable in high-cost scenarios

💡Sparsity is key to unlocking full potential of TTS, because unlike pretraining, where scaling shows diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths.

📄 Paper: https://t.co/hvfBpuTjwJ
🌐 Website: https://t.co/7HhzwASBI7
🔗 GitHub: https://t.co/YXzhsJ4RQ2
#ML #AI #LLM #Transformers #ScalingLaws #SparseAttention #Inference

[PyTorch @PyTorch]

Presenting HadaCore: Tensor Core Accelerated Hadamard Transform Kernel 🔍 Take a look at how we achieve state-of-the-art performance on NVIDIA A100 & H100 GPUs over Dao AI Lab’s Fast Hadamard Transform Kernel 🚄 Peak speedups of 3.6x—read more: https://t.co/KRISEdba6F #AI