Si-ze Zheng

ByteDance Seed & THU Postdoc | Previously @ DeepSeek AI | Visiting PhD @ UW | PhD @ PKU

[Zihao Ye @ye_combinator]

🚀 MLSys 2026 Contest - @nvidia Track is LIVE!
Registration is now open for the FlashInfer-Bench Challenge! Submit high-performance GPU kernels for cutting-edge LLM architectures on NVIDIA Blackwell GPUs.

Three Tracks
* MoE (Mixture of Experts)
* DSA (Deepseek Sparse Attention)
* GDN (Gated Delta Net)

Human experts AND AI agents welcome — evaluated separately. Let's see who builds the best kernels! 🤖
🎁 Prizes: Winners take home NVIDIA GPUs and are invited for presentation at MLSys 2026.
⚡ First 50 teams to register get free GPU credits from @modal - huge thanks for the sponsorship @charles_irl !

Whether you're a kernel wizard or building autonomous coding agents, we want to see what you've got.
🔗 Contest details: https://t.co/0ILK1D4Z9o

See you at MLSys 2026! 🔥

[Si-ze Zheng @DeeplyIgnorant]

Excited to share the latest Triton-distributed update: We’ve added support for Piecewise Mega EP MoE!
Thanks to token sorting + token saving techniques, it hits an algorithmic bandwidth of 200GB/s — surpassing theoretical limits 🚀

https://t.co/Hcq8mAtxFO

[Tianqi Chen @tqchenml]

🚀Excited to launch FlashInfer Bench. We believe AI has the potential to help build LLM systems . To accelerate the path, we need an open schema for critical workloads and an AI-driven virtuous circle. First-class integration with FlashInfer, SGLang and vLLM support👉

[Si-ze Zheng @DeeplyIgnorant]

🎉 Excited to share: We’ve open-sourced Triton-distributed MegaKernel! A fresh, powerful take on MegaKernel for LLMs—built entirely on our Triton-distributed framework.
https://t.co/lMbbYyd1uF

Why it’s awesome?
🧩 Super programmable
⚡ Blazing performance
📊 Rock-solid precision

[Overleaf @overleaf]

⚠️ Attention: The site is currently down. Our engineering team is investigating. We will update as soon as possible. You can track progress here: https://t.co/y7aRh5SBN4 Sorry for any inconvenience.

[Si-ze Zheng @DeeplyIgnorant]

🚀 We released Triton-distributed! 🌟
Build compute-comm. overlapping kernels for GPUs—performance rivals optimized libraries
🔗 https://t.co/lMbbYyd1uF
👏 Shoutout to AMD for testing our work! Check their blog:
🔗 https://t.co/xmZOOwwYG8

[Hanshi Sun @preminstrel]

🎉 Thrilled to announce our ShadowKV has been accepted to #ICML2025 as a ✨Spotlight Presentation❗️

❓Facing challenges with high-throughput long-context LLM serving? ShadowKV is here to help!

🚀 Achieves memory-efficient & high-throughput inference via sparse attention.
🌟 Minimizes memory footprint by storing low-rank keys on GPU & offloading values.
⚡ Boosts throughput up to 3.04x on an A100 with 6x batch size or 6x longer sequences without accuracy loss.

Discover how ShadowKV balances speed, memory, and precision for ultra-long contexts! 👇

📜 Paper: https://t.co/ZT7Xkw4BjL
🔗Blog: https://t.co/uOHLhHpuXc
💻 Code: https://t.co/sX2RzPWTMS

[Si-ze Zheng @DeeplyIgnorant]

@KuJerome Will be available soon

[Si-ze Zheng @DeeplyIgnorant]

Our paper "TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives" is accepted to the #MLSys2025 Conference held from May 12-15th in Santa Clara, California! Check out the full list of accepted papers here: https://t.co/jELiwlrNRh

[Zihao Ye @ye_combinator]

We are excite to announce FlashInfer v0.2!

Core contributions of this release include:
- Block/Vector Sparse (Paged) Attention on FlashAttention-3
- JIT compilation for customized attention variants
- Fused Multi-head Latent Attention (MLA) decoding kernel
- Lots of bugfix and improvements involving CUDAGraph compatibility, RMSNorm/RoPE numerical issue, etc.

blog post: https://t.co/tMBFmCfAc0

[NeurIPS Conference @NeurIPSConf]

NeurIPS acknowledges that the cultural generalization made by the keynote speaker today reinforces implicit biases by making generalisations about Chinese scholars. This is not what NeurIPS stands for. NeurIPS is dedicated to being a safe space for all of us. We want to address the comment made during the invited talk this afternoon, as it is something that NeurIPS does not condone and it doesn't align with our code of conduct. We are addressing this issue with the speaker directly.
NeurIPS is dedicated to being a diverse and inclusive place where everyone is treated equally.

[Zihao Ye @ye_combinator]

(1/4) Announcing FlashInfer, a kernel library that provides state-of-the-art kernel implementations for LLM Inference/Serving.

FlashInfer's unique features include:
- Comprehensive Attention Kernels: covering prefill/decode/append attention for various KV-Cache formats (Page Table, Ragged Tensor, etc.) for both single-request and batch-serving scenarios.
- Optimized Shared-Prefix Batch Decoding: 31x faster than vLLM's Page Attention implementation for long prompt large batch decoding.
- Efficient Attention for Compressed KV-Cache: optimized grouped-query attention with Tensor Cores (3x faster than vLLM's GQA), fused-RoPE attention, and high-performance quantized attention.

Check our blog and code at:
1. https://t.co/s3wnDg3uvj
2. https://t.co/52hP5U47dn

[Si-ze Zheng @DeeplyIgnorant]

Excited! Chat with Jeff @NeurIPS

[Si-ze Zheng @DeeplyIgnorant]

Glad to be informed that our recent paper "Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion" has been accepted by HPCA'23. This is our fourth paper based on TVM. Thanks to all the co-authors and TVM community.