UW SyFi

The Systems for Future Intelligence lab at @uwcse.

[UW SyFi @UWSyFi]

The hardest LLM serving bugs are the ones where it looked like everything worked, but accuracy quietly regressed.

Thrilled to share our latest work, Ekka, at ICML 2026! Ekka is an automated system for diagnosing silent errors in LLM serving frameworks. On real-world silent errors from vLLM/SGLang, Ekka reaches 80% pass@1 / 88% pass@5 diagnosis accuracy, improving over SOTA coding-agent baselines by 24–34%, and diagnosing 4 new bugs confirmed by developers. If you’re at ICML 2026, come say hi at Poster Session 5, Hall A #2405, on Wednesday, July 8, 5:00–6:45 PM KST.

Joint work between @UWSyfi and @awscloud with @MichaelGu341332, Zhen Zhang, Shaowei Zhu, Xinwei Fu, Jun Wu, Yida Wang, and @bariskasikci.

Paper 📗: https://t.co/FKzvD0oJ0J
ICML Page🌐: https://t.co/UlULRwqC9u
Blog: https://t.co/F9zvGoYwbj

[UW SyFi @UWSyFi]

📄 Paper: https://t.co/4Rfco6o25Q
✍️ Blog: https://t.co/YFeFsyeC05
🛠️ Trace and code: https://t.co/ZpNv3jYPbC
🌐 Live demo: https://t.co/IeahEcaDjC, where you can ask our chatbot questions about the trace, as well as analyze and explore your own traces.

This work was led by @KanZhu854772 , @mat_jacob1002, Chenxi Ma, @conlesspan, @thepadawang, @arvind_uw, and @bariskasikci!
(8/n)

[UW SyFi @UWSyFi]

🔥 Coding agents have become one of the hottest LLM workloads. But serving them looks nothing like serving a chatbot: 294× more input than output, hundreds of thousands of tool calls, and extremely long-tailed latency.

🚀 We are releasing the SyFI Coding Trace: ~4,300 real-world coding-agent sessions from our daily use, plus TraceLab, an open-source pipeline to collect, sanitize, analyze, and replay your own traces.
More in the thread below 🧵👇 (1/n)

[UW SyFi @UWSyFi]

🛠️ Our release includes trace examples, the collector/sanitizer, the full anonymized trace with analysis scripts and a chatbot, and a replay client for serving engines such as vLLM and SGLang.

🔍 This trace is an early look at real coding-agent traffic: self-driven loops, long-context short-output rounds, long-tailed tool execution, and imperfect prefix caching. It is also biased toward our own projects and habits, which is why we are releasing the full pipeline.

🤝 If you use Claude Code, Codex, or another coding agent, try TraceLab on your own logs, share a sanitized trace if you are comfortable, and help turn this first data point into a shared community resource. (7/n)

[Stephanie Wang @thepadawang]

M* is a new system for multimodal inference from our lab @uwsyfi. The system captures multimodal models as dataflow graphs, implements a generic engine for those graphs, and achieves SOTA results inference throughput/latency. Learn more here! https://t.co/q0PPpxFscC

[Keisuke Kamahori @KeisukeKamahori]

Excited to share that I’ll be interning at @nvidia this summer at the Santa Clara HQ, working on GPU architecture!

If you’re in the Bay Area, I’d love to grab coffee! Always happy to chat about agents, ML systems, GPU architecture, or anything in between.

#NVIDIALife

[Keisuke Kamahori @KeisukeKamahori]

New multimodal model architectures shouldn't require new serving systems.

Introducing our work, M* (M-Star): a universal serving system for multimodal models that separates what a model computes - a dataflow graph - from how it runs: placement, scheduling, batching, and transport.

Joint work across @uwcse, @StanfordAILab, and @CMU_ECE with Atindra Jha, Naomi Sagan, Irmak Sivgin, Rohan Sanda, @ste_veng, Mark Horowitz, @LukeZettlemoyer, Olivia Hsu, @jure, @bariskasikci, and @thepadawang.

[UW SyFi @UWSyFi]

Joint work between Megan Frisella, Shubham Tiwari, Andy Ruan, Yi Pan, Parker Gustafson, Mat Jacob, Gilbert Bernstein, Stephanie Wang at UW SyFI.

Check out our paper!

📄 https://t.co/hg7p5bGetc
📝 https://t.co/QItGXoZudG
💻 https://t.co/N4r2h10Qra

[UW SyFi @UWSyFi]

New distributed training strategies should not require new distributed runtimes.

Introducing Piper: a programmable PyTorch training system for deploying complex training strategies by separating model placement and GPU scheduling from model code.

📄 https://t.co/hg7p5bGetc

[UW SyFi @UWSyFi]

Unlike current frameworks, Piper correctly composes pipeline parallelism with ZeRO-2 and ZeRO-3 memory optimizations.

In our experiments on Qwen3 9B, Piper encodes correct sharding semantics and supports larger batch sizes where Megatron, DeepSpeed, and TorchTitan fall short.

[Mathew Jacob @mat_jacob1002]

If there’s one thing you should do to learn how to build performant systems in this AI era, it is following @KeisukeKamahori @sudopowr and @MichaelGu341332!