Dedong Xie

[UW SyFi @UWSyFi]

1/ We pointed VibeSys, our multi-agent system for designing systems, at Qwen3.5-397B-A17B on four AMD MI300As. No human wrote engine code.

In 105 hours, it went from a 12.8 tok/s PyTorch baseline to 2,242 tok/s goodput, 2.33× SGLang on the same hardware!

[UW SyFi @UWSyFi]

🔥 Optimizing LLM serving requires repeated experiments, and running them on real hardware is slow and costly.

Performance depends on how models, kernels, hardware, and serving policies interact. As models and workloads change, engineers must revisit their configuration choices.

🚀 Introducing ServingStudio: an integrated workbench for simulating, analyzing, and optimizing LLM serving systems.

🧵 More in the thread below. (1/9)

[Yi Pan @conlesspan]

Check out our latest work on coding agent serving led by @serendipity_zk ! If you want to analyze characteristics of coding agents, collect your own vibe coding data, or use traces to optimize serving, give it a spin!

[UW SyFi @UWSyFi]

🔥 Coding agents have become one of the hottest LLM workloads. But serving them looks nothing like serving a chatbot: 294× more input than output, hundreds of thousands of tool calls, and extremely long-tailed latency.

🚀 We are releasing the SyFI Coding Trace: ~4,300 real-world coding-agent sessions from our daily use, plus TraceLab, an open-source pipeline to collect, sanitize, analyze, and replay your own traces.
More in the thread below 🧵👇 (1/n)

[Keisuke Kamahori @KeisukeKamahori]

New multimodal model architectures shouldn't require new serving systems.

Introducing our work, M* (M-Star): a universal serving system for multimodal models that separates what a model computes - a dataflow graph - from how it runs: placement, scheduling, batching, and transport.

Joint work across @uwcse, @StanfordAILab, and @CMU_ECE with Atindra Jha, Naomi Sagan, Irmak Sivgin, Rohan Sanda, @ste_veng, Mark Horowitz, @LukeZettlemoyer, Olivia Hsu, @jure, @bariskasikci, and @thepadawang.

[Baris Kasikci @bariskasikci]

How to beat all compression using LLMs?

⚙️ Introducing LLMc — a lossless compressor built with LLMs. LLMc leverages the predictive power of LLMs to beat traditional compressors like Gzip and LZMA on natural language text. (1/4)
🔗 Blog Post: https://t.co/BFBfNr1zzS
💻 Code: https://t.co/0iXNFEnGo3

[Baris Kasikci @bariskasikci]

🎙️ Introducing VoxServe — a high-throughput, low-latency serving system built for Speech Language Models (TTS, STS, etc.), natively handling audio detokenization + streaming with performance as the core goal. (1/4)
🔗 blog post: https://t.co/WtX2GtsG0n
💻 code: https://t.co/JY2llp3epv

[Baris Kasikci @bariskasikci]

We present Nanoflow, an LLM serving framework with close-to-optimal throughput. Nanoflow aggressively overlaps compute, memory, network operations yielding 1.91x more throughput on Splitwise, Lmsys and ShareGPT datasets wrt the best baseline, Tensor RT LLM (~69% of optimal tput)

[Dedong Xie @DedongX]

@nihui 十年前西堤印象还不错，因为可以免费面包吃到饱（x

[Dedong Xie @DedongX]

@flightradar24 air China, or its predecessor CAAC.

[Dedong Xie @DedongX]

@Reuters RIP

[Dedong Xie @DedongX]

@nihui nihui快快好起来���ʕ⁠っ⁠•⁠ᴥ⁠•⁠ʔ⁠っ

[Dedong Xie @DedongX]

@nihui nihui生日快乐！

[Dedong Xie @DedongX]

@NASA @chandraxray @SOFIAtelescope So glad to see SOFIA making discovery even after retirement.

[Dedong Xie @DedongX]

@JustinTrudeau Just ordered the stamps of Canada post for the 50th anniversary of Summit Series😃

[Dedong Xie @DedongX]

@CcibChris Big shout out for Raptor!

[Dedong Xie @DedongX]

@GomezSource 🥳

[Dedong Xie @DedongX]

@n194at ANA? Their sea turtle series are the best in my mind.

[Dedong Xie @DedongX]

@DjsAviation Antarctica 😉

[Dedong Xie @DedongX]

@DjsAviation A350-1000