vLLM Conference is next week, and we have a packed schedule 🎊 📅. Here's the full list of events you should know:
Mon:
🔷 4–6PM vLLM × Ray × Google Cloud Happy Hour: https://t.co/Exw10SVIrV
🔷 6–9PM vLLM × Dynamo Meetup: https://t.co/id6TaOfIMs
Tue:
🔷 11–11:30AM vLLM Keynote from @simon_mo_
🔷 12–5PM vLLM Track Day 1
🔷 6–9PM vLLM × AMD Happy Hour: https://t.co/RVXTxGq0Lz
Wed:
🔷 12–5PM vLLM Track Day 2
🔷 6–8:30PM vLLM x DigitalOcean × NVIDIA Happy Hour: https://t.co/U1FSK6aYhZ
No ticket needed for the happy hours and meetups, but space is limited. To join the full event, register here: https://t.co/bTDMkOOwIR
Define your #AI evaluation strategy with precision. EvalHub collections let you set benchmarks, weights, and thresholds that match your deployment context, not generic leaderboard scores.
🔨 Build measurement criteria that answer your actual questions.
https://t.co/idcNPIDJWr
Speculators v0.5.0 ships DFlash training support. The Red Hat AI team trained a Gemma 4 31B speculator with it. It beats Eagle 3 and a standalone FP8 quantized verifier on inter-token latency. Combine DFlash with FP8 and the gains stack.
Serve it:
vllm serve -tp 2 RedHatAI/gemma-4-31B-it-speculator.dflash
https://t.co/i0ZmCqlwYO
This one has been in the works for a while. @cedricclyburn teaching LLM inference, compression, and benchmarking with @vllm_project -- free course with @DeepLearningAI. Proud of this one.
Happening by popular demand! Whether you’re new to vLLM or a long-time contributor, join us next Thursday, October 23, for a vLLM Office Hours a session focused on helping you build, contribute, and collaborate with the community. Get a calendar invite: https://t.co/X8hAHYR3rl
How do you make LLMs route queries to the right models or tools, efficiently and with low latency?
Join vLLM Office Hours on Oct. 9 to explore the new Semantic Router (https://t.co/tqHesjOXKd), a lightweight, high-performance router for @vllm_project.
https://t.co/X8hAHYRBgT
What's next for Red Hat AI?
Join our Red Hat AI leaders @joefern1, @KatarkiTushar, Steven Huels, & Jeff DeMoss for a Q4 sneak peek into the latest releases and key new features that help enterprises address their biggest challenges.
Sign up: https://t.co/4YjHTGdpH6
Big congrats to Red Hat’s newest @vllm_project committer, Nicolo Lucchesi (@nick_lucche)! 🎉
Nicolo already made major contributions:
⚡️ Optimized disaggregated serving with KV transfer + Nixl for faster heterogeneous tensor-parallel setups.
🗣️ Designed & implemented vLLM’s speech-to-text interface, powering v1/audio/transcriptions and v1/audio/translations.
🤖 Advanced TPU integration with optimized sampling, efficient compilation, and improved preprocessing.
Now he’s leading the charge on "Disaggregated Everything" - building a tokens-in, tokens-out interface to unlock the next wave of vLLM performance. 🚀
Congrats, Nicolo!
v0.10.2 marks the first release with official aarch64 support for vLLM! You can now install vLLM directly onto @nvidia 's GB200. Along with the PyPI release, our docker image is also multi-platform so pulling the right image just works.
More perf enhancements on the way!
🚀 Thrilled to announce GuideLLM v0.3.0!
This release is highlighted by a brand new Web UI, containerized benchmarking, and powerful dataset preprocessing.
GuideLLM GitHub: https://t.co/0iSPUOqmch
(Thread 👇)
Qwen3-Next dropped yesterday and you can run it with Red Hat AI today.
✅ Day-zero support in vLLM
✅ Day-one deployment with Red Hat AI
Step-by-step guide: https://t.co/ZjLJyfmMJm
The future of AI is open.
On August 23, we are co-hosting the Shanghai vLLM meetup with our friends at MetaX, so if you are in town and want to learn more about how the community is leveraging vLLM, then sign up and join in! @vllm_project https://t.co/UZKUq2L5jy
Happy 4th July!
Speed is the Moat & @AnushElangovan & his team Keeps Running Faster & Faster. Still lots of areas where ROCm has gaps but many are already closing
Join us: IMDA x Red Hat Technical Sharing Session!
🗓️ Thursday, 19 June 2025, 2.30pm @ Red Hat SG Office
⚡High-performance inference with vLLM
⚙️ Deploying LLMs in resource-limited environments
🌐 Scalable GenAI across hybrid cloud with llm-d
Register: https://t.co/XTgT4ZXePu
Thank you @AMD@LisaSu@AnushElangovan for Advancing AI together with @vllm_project! We look forward to the continued partnership and pushing the boundary of inference.
🇯🇵 Join us for an in-person @vllm_project meetup on Monday, June 16 in Tokyo. Or tune in via live stream!
Agenda:
-Intro to vLLM
-Japanese LLM adoption
-Model optimization w/ LLM Compressor
-Distributed inference w/ llm-d
-Q&A and lightning talks
RSVP: https://t.co/ar7J22GvsF