@linf1017 Thank you for the shoutout! Yes, the original core of the code (version 0.1.0) was mostly written by hand and never fully finished, but in the last 2 months I’ve revisited it and decided to experiment with agentic development on it. I’m quite happy with the result!
I also want to highlight one code called Mallard solver, which is based on TENO and TENO-E reconstructions from 3rd to 6th order and GFM near-wall turbulence modeling (WMLES). It is indeed developed with assistant of Claude AI code:
https://t.co/xMjZVqu3eX
High throughput on the grill. Zero latency on the drinks.
vLLM maintainers and friends gathered during the PyTorch Conference for an evening of Brazilian steak, drinks, and great conversation! 🥩🥂
Grateful for this community and everyone who keeps vLLM moving forward. 🚀
1/ GLM 5.3 at full 1M context on a single 8xH200 node. That didn't fit before.
Agentic serving is why it's hard: many concurrent requests, each with a long context that keeps growing. The GPU block pool is fixed, so the KV cache runs out of room. Something has to give.
Sparse MLA only attends to the top-K tokens, so the rest of the KV need not live on the GPU. Hybrid HiSparse in vLLM builds on that, and a request keeps decoding after its KV stops fitting in HBM.
It keeps KV on the GPU while there is room. Under pressure a request releases its coldest pages to host memory, keeps a small hot buffer of what the indexer asks for, and keeps decoding instead of being preempted.
📊 Demonstrated on GLM 5.3, one 8× H200 node, full 1M context. Same host memory, configured concurrency 32: KV offloading kept 5-6 requests running. Hybrid HiSparse kept 19-25.
🔹 Hot pages are ordinary KV blocks from the same pool (Hybrid Memory Allocator)
🔹 One fused kernel resolves resident, hot and missing rows, CUDA-graph capturable
🔹 Prefix caching, OffloadingConnector, P/D imports and MTP keep working
Built by @RedHat_AI and @PrimeIntellect with the vLLM community. Planned for v0.30; pinned commit, flags and calculator are in the post👇
🔗 https://t.co/6y7u5Hfc8c
~4x faster Kimi-K3 decoding.
Our new DSpark speculator takes single-stream interactivity from ~110 to ~435 tok/s/user on math reasoning, delivers ~3.5x the output throughput at matched interactivity under load, and thanks to sliding window attention (2048-token window across all 5 draft layers), our model holds steady acceptance out to 20K context across 10 LongBench domains.
Check it out! https://t.co/AGQlV61Y8U
More details in reply:
Throwback to yesterday, with our very own @MatthewBonanni in his PhD Defense!
We congratulate him for this tremendous achievement and wish him all the best for the future!
My other publication from the #40thISOC is now online!
We studied the distribution of residence time in a scramjet cavity and developed some low-order models. Using these, we showed that the skewed shape of the distribution improves combustion stability.
https://t.co/WeD7rlqTZV
Happy to share that my recent work on rotating detonation engines is now online!
We studied how the inhomogeneous composition in the chamber distorts the detonation and how this distortion couples back to the inhomogeneity ahead of the next passing wave.
https://t.co/pdd5N8uxgW
Yesterday I had the pleasure of presenting at ICNC 2024 in Kyoto, Japan, continuing my work on adaptive combustion modeling in scramjets.
@Combustion_Intl
Excited to have our lab members present our research at ICNC in Kyoto, Japan!
Here’s our PI, Matthias, in his presentation on Machine Learning for Combustion.
@Combustion_Intl
Rotating detonation engines are next-gen propulsion systems that promise better thermo efficiency 🚀
Check out our new paper (coauthored by @MatthewBonanni) where we performed high-fidelity sims and developed a wavenumber analysis method.
Link: https://t.co/7FqhU7rWuE
Introducing a new simulation framework that leverages advances on @TensorFlow and TPU architecture to enable efficient large-scale simulations of turbulent flows (e.g., for cloud formation, wildfires, and more) at unprecedented scale. Learn more at https://t.co/axz1own3Xm