Yi Pan

Incoming PhD Student @BerkeleySky | Prev @UWSyFI @sjtu1896

[Guilherme Favaron @guifav]

Every team shipping a coding agent — Claude Code, Codex, Cursor — is really running a serving-systems problem. The "tech behind the tech" is the LLM-serving stack underneath, and until now nobody had real data on what that workload looks like.

New arXiv (2606.30560) from @bariskasikci's SyFI lab (@UWSyFi, @uwcse) is the first large cross-provider trace of real coding-agent use: ~4,300 sessions, 350K LLM steps, 430K tool calls, 43 developers, 8 months, Claude Code + Codex.

It breaks the intuition that agents mean long generations. The median step replays ~119K context tokens to emit just ~214 output tokens — two orders of magnitude more reading than writing. So the bill is the context, not the answer: prefix tokens are 59.5% of total cost.

Tool calls are brutally long-tailed: 80+ tools, but the top 3 are 80%+ of calls, and the 4% of calls that run >1 min eat 85% of all tool time.

And the prefix cache everyone leans on? 95.7% hit rate — yet misses cluster right after a human pauses to think, amplifying prefill 3.8x. Those human-gap misses alone are ~46% of fresh tokens and ~13% of spend.

For technical leaders: your agent's cost and latency live in the loop, the replayed context, and the idle gaps — not raw token generation. Tune tool-call overhead, append-length-aware prefill, and KV-cache eviction around human gaps before you scale the fleet.

[Mathew Jacob @mat_jacob1002]

To make serving coding agents more efficient, we need better hills to climb than traces from synthetic benchmarks like SWE-Bench. This work led by @KanZhu854772 is critical in curating a realistic workload for the ML Sys community to climb! (plus you can analyze your own usage!)

[Yilong Zhao @ylzhao_dreamer]

Checkout this great statistic and analysis on modern agentic inference workloads!

[Yi Pan @conlesspan]

Check out our latest work on coding agent serving led by @serendipity_zk ! If you want to analyze characteristics of coding agents, collect your own vibe coding data, or use traces to optimize serving, give it a spin!

[UW SyFi @UWSyFi]

🔥 Coding agents have become one of the hottest LLM workloads. But serving them looks nothing like serving a chatbot: 294× more input than output, hundreds of thousands of tool calls, and extremely long-tailed latency.

🚀 We are releasing the SyFI Coding Trace: ~4,300 real-world coding-agent sessions from our daily use, plus TraceLab, an open-source pipeline to collect, sanitize, analyze, and replay your own traces.
More in the thread below 🧵👇 (1/n)

[Yi Pan @conlesspan]

@xiangfeng_zhu @ratulm @arvind_uw Congrats!

[Yi Pan @conlesspan]

Check out our recent work!

[UW SyFi @UWSyFi]

New distributed training strategies should not require new distributed runtimes.

Introducing Piper: a programmable PyTorch training system for deploying complex training strategies by separating model placement and GPU scheduling from model code.

📄 https://t.co/hg7p5bGetc

[Yi Pan @conlesspan]

@cHHillee This is related to https://t.co/WYcHQbJBA3. My current workaround is using a script to manually adjust the timestamps to ensure no overlapping 🫠

[Yi Pan @conlesspan]

Huge congrats to the team!!!

[Baris Kasikci @bariskasikci]

Super stoked that UW SyFI (https://t.co/GsIZJi5LB5) members won a number of prizes at the MLSys'26 competition, NVIDIA Track. Hugre congrats to @KeisukeKamahori , @sudopowr , Yile Gu, Wei Shen, Steven Gao! Thanks to @nvidia , @modal , and the Flashinfer team for the support.

1st place in the GDN Track — Full-Agent Approach
2nd place in the GDN Track — Agent-Assisted Approach
3rd place in the DSA Track — Full-Agent Approach

[Yi Pan @conlesspan]

@ZhiyuanZeng_ GOAT!

[Vic Shihang Li @sudopowr]

Today's AI agents can diagnose production incidents, but they start from scratch every single time. What if they could remember?
New on @acmsigops: our work on the Self-Defining Operator, a multi-agent system with long-term memory for autonomous ops.

[Yi Pan @conlesspan]

@ying11231 @radixark @lmsysorg @slime_framework @jxwuyi @xai Congrats!

[Baris Kasikci @bariskasikci]

How to beat all compression using LLMs?

⚙️ Introducing LLMc — a lossless compressor built with LLMs. LLMc leverages the predictive power of LLMs to beat traditional compressors like Gzip and LZMA on natural language text. (1/4)
🔗 Blog Post: https://t.co/BFBfNr1zzS
💻 Code: https://t.co/0iXNFEnGo3

[Baris Kasikci @bariskasikci]

🎙️ Introducing VoxServe — a high-throughput, low-latency serving system built for Speech Language Models (TTS, STS, etc.), natively handling audio detokenization + streaming with performance as the core goal. (1/4)
🔗 blog post: https://t.co/WtX2GtsG0n
💻 code: https://t.co/JY2llp3epv

[Yi Pan @conlesspan]

Congrats!

[Xingyang Li @XYLi_Bruce]

Thrilled to announce that my first first-author paper in efficient ML is accepted by #NeurIPS2025! Let’s make video generation bigger and greater!

Thanks my mentors and my advisor for their kind mentorship and encouragement. Can’t wait to see you guys at San Diego!

[Yi Pan @conlesspan]

@b1antaidaye 😭

[Yi Pan @conlesspan]

@iskyzh Just had my dinner there😋

[Yi Pan @conlesspan]

@bariskasikci @emnlpmeeting Congrats, Baris and Keisuke!

[Tianyin Xu @tianyin_xu]

A petition to SIGOPS to adopt the USENIX Annual Technical Conference (ATC) and retain its steering committee
https://t.co/OleG8sYYRl

(not sure whether it can be done by SIGOPS alone, but it's great to let the voice be heard)

[Yi Pan @conlesspan]

@bariskasikci congrats!

[Yi Pan @conlesspan]

@BanghuaZ congrats!