WenxuanXu

ElastixAI tpsmaxxxxing！

[WenxuanXu @LesileXu]

#sglang #inference coooool

[SGLang @sgl_project]

Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.

Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.

Register: https://t.co/7C6TzrkJYV

[WenxuanXu @LesileXu]

@uncle_moon_x +1s

[WenxuanXu @LesileXu]

@sheriyuo chi有些东西和意林一样

[LMSYS Org @lmsysorg]

DeepSeek V4 by @deepseek_ai just dropped! SGLang is ready on Day 0 with a full stack of optimizations from architectures to low-level kernels. We also deliver a verified RL training pipeline in Miles (by @radixark) for V4 at launch:

1️⃣ Native "ShadowRadix" Design: DeepSeek V4's hybrid attention is complex. Our new ShadowRadix engine is the first to provide native prefix caching for SWA and compressed KV pools, making 1M+ context retrieval seamless and memory-efficient.

2️⃣ High-Performance Kernels:
- Flash Compressor: IO-aware fused kernels, 10x faster than naive implementations.
- Lightning TopK: High-speed indexing for 1M context in just 15µs.
- Integrate FlashInfer trtllm-gen MoE, FlashMLA, and MegaMoE kernels

3️⃣ Rich Features: Speculative decoding, HiSparse, Attention DP/TP/CP and MoE TP/EP, and multi-platform support

4️⃣ Verified RL: The open-source RL pipeline: full parallelism (DP/TP/EP/PP/CP), tilelang kernels, tensor-level checked precision, verified with growing reward.

Get started immediately with our out-of-the-box Cookbook 👇
Enjoy! #DeepSeekV4 #SGLang #LLM

[Dan Fu @realDanFu]

📢 Super excited to announce Parcae! We've been thinking about scaling laws and the "right" way to get more FLOPs.

Turns out layer looping - with the right parameterization - gives you a new axis to scale!

Parcae matches Transformers 2x their size (w/ the same data), and outperforms prior formulations of looped models.

But - you need the right parameterization to get these gains against strong Transformer baselines. Looped models are famously unstable to train, with tons of loss spikes and hyperparameter sensitivity.

The main technical challenge with looped models is residual explosion - if you're passing the activations through the same layers over and over, some otherwise benign parameterizations cause huge instability.

Our key idea: we can think of the residual stream of a model as a time-varying dynamical system - the same fundamentals behind SSMs like Mamba and S4. Then a few modest modifications to classic Transformers (stable diagonalization of injection params, LN before embeddings) can stabilize the looped models. The resulting models are more stable to train, but also reach higher quality.

It's strong enough to start to derive new scaling laws. Classically - we know you need to scale parameters with data to be FLOP-optimal. With Parcae, we find a third axis - given fixed parameters, you additionally want to scale FLOPs by looping as you scale data.

Super excited to see how these ideas hold, and what we can do with looped models!

Check out @hayden_prairie's great explainer thread below, and see links for our paper, blog, and models. Joint w/ @zacknovack and @BergKirkpatrick, and a fun collab between @togethercompute and my lab at @ucsd_cse. Enjoy!

[WenxuanXu @LesileXu]

@aclmeeting Great conference👍👍

[WenxuanXu @LesileXu]

@FUZxxl @tux0r @michaelfreedman sounds like nvidia dev tech. got the same before

[WenxuanXu @LesileXu]

@LuckyJoe198x @grok 总结这本书的内容

[WenxuanXu @LesileXu]

@tuturetom @karpathy @readwise save

[WenxuanXu @LesileXu]

哎

[WenxuanXu @LesileXu]

@liin1211 优秀

[WenxuanXu @LesileXu]

@liin1211 @Larry36719357 oh！Big Con！

[WenxuanXu @LesileXu]

@qian_pin 早点休息

[WenxuanXu @LesileXu]

@qian_pin fine pin

[WenxuanXu @LesileXu]

@XiangHCI 生日快乐喵！

[WenxuanXu @LesileXu]

@liin1211 学长好厉害！

[WenxuanXu @LesileXu]

@TheGleGod 你去？

[WenxuanXu @LesileXu]

@liin1211 好羡慕