Great write-up from the @anyscalecompute team on PD disaggregation with Ray Serve + vLLM! PD Disagg is one of the most difficult techniques to get right in serving; the wins are real, but only in the right settings.
Great to see it pressure-tested on AMD MI325X with Ray Serve + vLLM!
🎉 llm-d v0.7 is officially live! 🚀
Earlier releases proved what llm-d could do, v0.7 is about making sure you can easily deploy it in production.
Backed by a massive 3.5x surge in community PR volume, this release hardens the stack for serious scale.
https://t.co/fY1du9Tudg
📘Our latest paper in @Applsci Special Issue “Artificial Intelligence and Data Science”
“Benchmarking Ollama and vLLM for Concurrent LLM Serving”
👩🏼🏫 by Betül Ay and Yunus Emre Demirdağ
🔗The results on NVIDIA H100 GPU (Qwen3-4B): https://t.co/c9aYEAwb5o