Rui Pan 潘瑞

4th-yr PhD @PrincetonCS working on efficient LLM inference, previously @Google @AmazonScience @maxplanckpress @WisconsinCS

[Rui Pan 潘瑞 @ruipeterpan]

@xiangfeng_zhu @ratulm @arvind_uw Congratulations!!! 🛫

[Rui Pan 潘瑞 @ruipeterpan]

@YichuanM @JiaZhihao Congrats!

[Zhihao Jia @JiaZhihao]

🚀Introducing Motus, the open-source agent infrastructure that learns in production.

Existing agent infra serves static agents: the harness, model, and workflow are fixed after deployment. But static agents degrade over time. The harness goes stale, new models go unincorporated, context drifts, and latency compounds.

Motus closes this gap by learning from every trace (failures, latency, cost, and task outcomes) and using those signals to continuously optimize agent harness, model orchestration, context memory, and end-to-end latency.

Early results: higher accuracy than any single frontier model at 2.3× lower cost (Terminal-Bench 2.0, SWE-bench Verified), with 52% lower latency and 45% better memory recall.

Open source under Apache 2.0. Works with any agent SDK. Deploy with one command.

https://t.co/C4u6JUzige
https://t.co/QIfKIikZQb

[Zhiqiang @Zhiqiang_Xie]

New hierarchical memory optimization out! This time for sparse attention! Try it out and stay tuned for more Hi-series : )

[Rui Pan 潘瑞 @ruipeterpan]

@vikhyatk @srush_nlp We had an MLSys paper on this — hope you find it interesting!

https://t.co/tdinnMpU0n

[Rui Pan 潘瑞 @ruipeterpan]

@charles_irl @rahulgs @akshat_b @_dcw02 @modal We wrote a concurrent work that also explored the policies around using dLLMs as drafters: https://t.co/g9IJbYNmD5. TLDR: dLLM’s acceptance rate is not necessarily higher, but its ability to unmask multiple tokens in each forward pass allows them to generate longer drafts cheaply

[yifei e/λ (meetmeinshibuya Sept 13) @yifever]

congrats to llama 3 large for winning the LLM trading contest by not participating

[Rui Pan 潘瑞 @ruipeterpan]

Super cool work!

[Jiawei Zhao @jiawzhao]

Introducing DeepConf: Deep Think with Confidence

🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens.

It also delivers many strong advantages for parallel thinking:
🔥 Performance boost: ~10% accuracy across models & datasets
⚡ Ultra-efficient: Up to 85% fewer tokens generated
🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!)
⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below)

📚 Paper: https://t.co/jnBnRzQczh
🌐 Project: https://t.co/kGq1kATTu0

joint work with: @FuYichao123 , xuewei_wang, @tydsh

(see details in the comments below)

[Lijie(Derrick) Yang @LijieyYang]

[1/N] 🚀 Excited to introduce my first work at @Princeton:
LessIsMore – a training-free sparse attention method tailored for efficient reasoning in LRMs, achieving lossless accuracy with high sparsity up to 87.5% and 1.1x avg decoding speedup compared to Full Attention on reasoning tasks like AIME-24. (More details in 🧵)

💻 Code: https://t.co/5e4QC2A6S2
📄 arXiv: https://t.co/yXyTqRjPe0
🔍 HF Daily Paper: https://t.co/yXyTqRjPe0

[Siddhant Ray @siddhantrayyy]

With RAG and agents becoming ubiquitous in LLM systems, tuning quality and performance JOINTLY is essential to achieve the best LLM quality-of-experience.

Our paper at SOSP this year, addresses this exact tradeoff!🔥

[Lindia Tjuatja @lltjuatja]

committed to doing my part in decreasing reviewer workload by writing fewer papers

[Rui Pan 潘瑞 @ruipeterpan]

@Synced_Global @wzihanw It’s supposed to be @rui4research lol

[Arthur Zucker @art_zucker]

A quick update on the future of the `transformers` library!

In order to provide a source of truth for all models, we are working with the rest of the ecosystem to make the modeling code the standard.

A joint effort with vLLM, LlamaCPP, SGLang, Mlx, Qwen, Glm, Unsloth, Axoloth, Deepspeed, IBM, Gemma, Llama, Deepseek, microsoft, nvidia, internLM, Llava, AllenAI, Cohere, TogetherAI.....

[Rui Pan 潘瑞 @ruipeterpan]

@melissapan Thanks, Melissa!

[Rui Pan 潘瑞 @ruipeterpan]

I'm too lazy to write a real promo post or make a poster. But! 🤓👆I will be at MLSys presenting our work this Thursday in Session 10. This stemmed from an amazing collaboration with Zhuang, Zhen, Can, @ZancatoLuca, @yidawang from AWS, and @tri_dao and Ravi from Princeton. Come say hi -- happy to chat!

[𝚐𝔪𝟾𝚡𝚡𝟾 @gm8xx8]

Marconi: Prefix Caching for the Era of Hybrid LLMs

Marconi improves caching for hybrid LLMs with policies optimizing reuse likelihood and compute savings, achieving 34.4× higher token hit rates and significantly reducing latency.

[Tri Dao @tri_dao]

Congrats to @ruipeterpan and the entire team for the recognition of Marconi at MLSys. Hybrid mamba-transformer LLMs are getting more popular, and the change in compute / cache size ratio changes the way you should cache and schedule requests

[Rui Pan 潘瑞 @ruipeterpan]

@tri_dao Thanks, Tri! It was a pleasure to work together on this project and I couldn’t have asked for a more awesome collaborator 🥹

[Rui Pan 潘瑞 @ruipeterpan]

@ye_combinator @lmsysorg Congrats 🎉

[Rui Pan 潘瑞 @ruipeterpan]

@LijieyYang Thanks, Lijie!

[Rui Pan 潘瑞 @ruipeterpan]

@FuYichao123 Thanks, Yichao! ☺️