Chien-Yu Lin

PhD student in UW CSE

[Chien-Yu Lin @cylinbao]

@YichuanM Wow cleaver and neat!

[Chien-Yu Lin @cylinbao]

@yi_xin_dong @tqchenml Congrats! It’s a super cool work!

[Shanli Xing @shanli_xing]

🚀Meet flashinfer.sampling—our sorting-free GPU kernels for lightning-fast #LLM sampling.

Our implementation achieves over 50% reduction in sampling time.

Blog post: https://t.co/R780Rth03x

[Zihao Ye @ye_combinator]

We are excite to announce FlashInfer v0.2!

Core contributions of this release include:
- Block/Vector Sparse (Paged) Attention on FlashAttention-3
- JIT compilation for customized attention variants
- Fused Multi-head Latent Attention (MLA) decoding kernel
- Lots of bugfix and improvements involving CUDAGraph compatibility, RMSNorm/RoPE numerical issue, etc.

blog post: https://t.co/tMBFmCfAc0