Zhichen Zeng

RL & intelligent system @radixark | ph.d. @uw; b.s. of physics @ustc; previous intern @bytedance seed, @msftresearch

[Zhichen Zeng @zz30gs]

Congratulations to @thinkymachines on the release of the powerful Inkling model and fine-tuning on Tinker!

SGLang (https://t.co/61TheQnmGf) and Miles (https://t.co/gYSY8p6Ib9) provide Day-0 support across serving and reinforcement learning.

Very excited to have built a end-to-end validated RL pipeline backends for Inkling in Miles, supporting full-parameter and LoRA RL across both text-only and multimodal reasoning tasks. Across these runs, train–rollout mismatch remains small while reward and evaluation performance improve consistently. Feel free to try it out and customize Inkling with either full-parameter training or LoRA!

It was a pleasure working with Ke, @ChengWan17, @BanghuaZ, @ying11231 and all of SGLang teammates in @radixark; and huge thanks to @aurickq and our collaborators at @thinkymachines for making this Day-0 launch possible!

SGLang and Miles Day-0 support blog: https://t.co/WYAxDr9Rd7

[Thinking Machines @thinkymachines]

Today, we are introducing Inkling.

Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.

https://t.co/Ghebq5mG30

Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

[Zhichen Zeng @zz30gs]

@zhaoran_wang bbzl

[Banghua Zhu @BanghuaZ]

SGLang delivers a good speed for K3 at day-0. Behind it were many sleepless nights across the SGLang ecosystem, and an incredible relay race across the ecosystem.

@nvidia and @AMD put serious engineering power into kernels and hardware enablement. @KVCache_AI pushed support across different hardwares, PD disaggregation, and HiCache.

Then @modal jumped straight into co-development and trained their ridiculously fast Dflash model. (They’ve already reported 460 tok/s. Huge congratulations!)

@baseten tested the SGLang implementation, gave us invaluable feedback, and submitted PRs to improve tool calling. @digitalocean provided compute when we needed it.

Meanwhile, @Kimi_Moonshot, @googlecloud, @nebiustf, @fal, @runpod, @DeepInfra, @gmi_cloud, and many others we can’t name yet were racing to test, optimize, and support Kimi K3 with SGLang.

That’s the magic of open source: not one team doing everything, but many teams picking up the baton and running with it.

And for a little extra fun, we used K3 to create the entire launch video and its companion game. AI does make life much easier lmao.

[Ying Sheng @ying11231]

The game (made by Kimi K3) is a bit too fast in showing how many optimizations have been added to get there 😂

Drop us a message if serving K3 is something you need!

[Zhichen Zeng @zz30gs]

@zhzHNN @ytwang_13 😁

[Zhichen Zeng @zz30gs]

@zhzHNN Always learning from the Moon Uncle! 🌕

[Zhichen Zeng @zz30gs]

@aurickq @sgl_project @radixark @vllm_project @inferact @lightseekorg @huggingface @UnslothAI @togethercompute @FireworksAI_HQ @baseten @modal @nvidia 🐐🐐

[Zhichen Zeng @zz30gs]

@YujieZhao455906 Thanks!!!

[Zhichen Zeng @zz30gs]

@lilianweng Congrats!

[Zhichen Zeng @zz30gs]

Let's gooo miles!!!

[RadixArk @radixark]

Weeks of work with @thinkymachines to make sure Inkling is trainable on day one.

Open model, open infra, open recipes: LoRA to full parameter, all in the Miles cookbook 👇

[Banghua Zhu @BanghuaZ]

Big fan of TML’s mission to build AI that extends human will and judgment, and reimagine how humans and AI collaborate.

With Inkling, released with full weights and built for customization, we see a great future of US open models. Congrats, @thinkymachines!

[Mira Murati @miramurati]

Today we share the worldview behind our mission.

Human values don't average out. Local knowledge can't be centralized. The good future has many AIs, raised in different places, shaped by the people they serve, disagreeing with each other the way we do.

https://t.co/A14SurOM2K

[LMSYS Org @lmsysorg]

🚀 Accelerating SGLang HiCache with Netpreme X-Mem™ MPU: new blog from @netpreme on plugging a TB/s memory tier into HiCache's hierarchical KV cache!

In multi-turn coding sessions (Claude Code on SWE-bench), prefix-cache hit rates average ~98%. At this point, KV bandwidth is the bottleneck, not compute. Swapping Host DRAM for X-Mem™ as the offload tier:
✅ Up to 6.7× lower TTFT 
✅ 33–50% higher TPS/user under load
✅ 30% higher system throughput
✅ Drop-in via HiCache's tiering interface

[Zhichen Zeng @zz30gs]

@zdhnarsil 🐮🍺

[PyTorch @PyTorch]

Built on PyTorch, Ray, SGLang, and NVIDIA Megatron-LM, Miles is an open source framework from RadixArk for large-scale LLM reinforcement learning post-training.

Miles uses PyTorch for models, numerics, profiling, and extensibility; Ray for orchestration; SGLang for rollout generation; and Megatron-LM for distributed training.

The framework supports asynchronous rollout and training, NCCL/RDMA weight synchronization, MoE-aware rollout/training alignment, low-precision recipes, LoRA, fault tolerance, observability, and extension points for custom algorithms and model architectures.

🔗 Read more in our latest blog from the Miles Team: https://t.co/fekP7rOoH7

[UW SyFi @UWSyFi]

🔥 Coding agents have become one of the hottest LLM workloads. But serving them looks nothing like serving a chatbot: 294× more input than output, hundreds of thousands of tool calls, and extremely long-tailed latency.

🚀 We are releasing the SyFI Coding Trace: ~4,300 real-world coding-agent sessions from our daily use, plus TraceLab, an open-source pipeline to collect, sanitize, analyze, and replay your own traces.
More in the thread below 🧵👇 (1/n)

[Lilian Weng @lilianweng]

A super long overdue (3+ years?) post on scaling laws.

Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.

The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.

https://t.co/HP26eJvjHB

[slime @slime_framework]

Thrilled to share the release of GLM-5.2 🎉

It is the result of tremendous effort from the whole team, and our best model so far — stronger coding, usable 1M context, and further progress on long-horizon / agentic tasks.

For slime, we are honored to once again support RL and OPD at a new scale behind GLM-5.2.

As the official post says, intelligence should be open, accessible, and ready to build with. We believe post-training capabilities should be the same.

That’s why slime merged GLM-5.2 support on day 0, so the community can more easily explore, reproduce, and build with GLM-5.2.

PR: https://t.co/k9C0bhPoCG

[LMSYS Org @lmsysorg]

🚀 New blog: The next generation of speculative decoding: DFlash and Spec V2

DFlash + Spec V2 hit >4.3X baseline throughput for LLM inference, now the default speculative decoding engine in SGLang! Together with @modal and https://t.co/ZXetBKIRym, our jointly-released DFlash drafter for Qwen 3.5 397B-A17B beats both baseline and native MTP in every setting we benchmarked:
1️⃣ >4.3X baseline & 1.5X native MTP throughput (concurrency 1, HumanEval, 8xB200)
2️⃣ Block diffusion drafter: a full token block in one forward pass
3️⃣ KV injection: target-model features fed into every draft layer’s KV cache for higher acceptance
4️⃣ Spec V2 overlap scheduler: +33% end-to-end

Read the code, deploy a DFlash server, and start experimenting!

[Zhichen Zeng @zz30gs]

@JxlDragon 😍😍