So happy @kevin_flansburg will be joining us at the first SGLang Summit! We’ve been working closely with the Cloudflare team for a long time, and they’ve been such a great team to work with. Looking forward to hearing more about their work and catching up in SF! 😃
I’ve been working towards AGI my whole life, and as we enter this pivotal moment, I’m stepping into a new role as Chair of Google DeepMind & Chief Scientist of Alphabet. This will allow me to focus on long-term strategy, and accelerating scientific breakthroughs, including leaning into my work at Isomorphic to help cure disease.
I’m excited that @koraykv will be stepping up to lead GDM as SVP, alongside @joshwoodward and our exec team. I could not be more excited and confident about our amazing next chapter! 🚀
https://t.co/2WtlIIlTUa
RadixArk's mission is to make frontier AI infrastructure open and accessible to every builder. SGLang embodies that, and we're excited to work with Google Cloud @googlecloud@googledevs to bring its performance and flexibility to the TPU ecosystem.
RadixArk has proudly co-signed this letter.
Our mission is to make frontier AI infrastructure open and accessible to everyone. We power the open-weight ecosystem with SGLang for serving and Miles for training.
Excited to build the future of AI together with the community!
Jensen's first post and it is a letter on why open models matter.
Working in open source inference, I see the diffusion argument play out daily. Teams across finance, telecom and automotive are moving open models into production on NVIDIA hardware right now, many of them in places a closed API could never reach for sovereignty or compliance reasons.
The world needs both frontier closed and frontier open models. Could not agree more. Welcome to X @JensenHuang and thank you for putting the weight of NVIDIA behind this!
Quantum computing has always felt “10 years away.”
Now, experts are saying it’s closer than you think. Here are 10 terms to help you understand what’s coming next: https://t.co/ixbBcVxQwL
Big fan of TML’s mission to build AI that extends human will and judgment, and reimagine how humans and AI collaborate.
With Inkling, released with full weights and built for customization, we see a great future of US open models. Congrats, @thinkymachines!
🎉 SGLang v0.5.15 is out!
We spent this cycle tuning GLM-5.2 NVFP4 for production serving, now hitting 500+ tok/s/user on 8x B300 and 450 on 4x GB300 (bs=1).
We will put commands to run this at the thread below, and full technical details and instructions on a blog very soon 🫡
And we have some newly supported models: Hunyuan 3 (Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, Baidu Unlimited-OCR, JoyEcho, and Qwen3.6.
Here are highlights for this release:
- Breakable CUDA Graph is now the default capture path
- Native web search built in, powered by @ExaAILabs
- Decode context parallelism for MLA models, including DeepSeek V3
- FlashInfer all-to-all for routed MoE
- DeepSeek-V4: FlashMLA sparse prefill now on by default (>10% throughput on long context), plus a non-paged indexer for long-context prefill (>5% e2e)
We welcomed 43 new contributors, and thanks again for our amazing partners and model makers: @NVIDIAAI@AMD@intel@Zai_org@TencentHunyuan@Alibaba_Qwen@deepseek_ai@Sapient_Int
Now. MAX LOAD! MAX OUTPUT! 🚀
Miles is now featured on the PyTorch Foundation blog.
As models grow, shift from dense to MoE, and span more specialized hardware, RL post-training is no longer just about the algorithm. It is a distributed systems problem. Miles is our open-source RL training framework, built for exactly that.
It comprises four systems behind a small, pluggable trainer: SGLang (@sgl_project) for rollout, Megatron-LM (@NVIDIAAI) for training, Ray (@raydistributed) for orchestration, and PyTorch (@PyTorch) as the common layer for models and numerics.
Out of the box, you also get MoE-aware rollout/training alignment, a unified BF16/FP8/MXFP8/NVFP4/INT4-QAT pipeline, fast NCCL/RDMA weight sync, fault tolerance, and ready-to-run recipes for frontier models like DeepSeek V4, GLM 5.2, Qwen3.6, Kimi K2.6, and Nemotron 3 Ultra.
Our goal is simple: make frontier-scale LLM RL easier to reproduce, extend, and operate.
Thank you, PyTorch Foundation, and everyone who got Miles here, especially the legendary @slime_framework team!
Day-0 support is the starting point, not the finish line. The SGLang community has always been turning frontier models into fast, production-ready serving systems as quickly as possible.
In 2 months, folks achieve 5X throughout speedup in Nvidia GB300.
Proud of the @sgl_project and @NVIDIAAI teams pushing this forward.