halfrost

✨ Fiat Lux ✨ Go Bears 🐻💙💛 Alumni @UC Berkeley @CMU 🇺🇸 Bay Area
🧑🏻‍💻 Computer Science❤️github@halfrost

[halfrost @halffrost]

助力一下 gopher 今年金九银十跳槽季，小弟我把去年刷的 leetcode 整理出了 520 题汇成了小册子，全部都是 go 实现的 runtime beats 100% 了。在线阅读地址：https://t.co/G14GhATFhe，网页支持了 PWA，可以像 mac 应用一样沉浸式阅读。@chaishushan
@changkun @talkgo_night @unknwon @astaxie

[halfrost @halffrost]

I traced vLLM V1’s KV cache from GPU budget to physical slot: fixed-size blocks, prefix hashes, refcounts, preemption, and block tables. Part 6: https://t.co/tJGZLH1n52

[halfrost @halffrost]

vLLM V1 has no global “prefill phase” or “decode phase.”

Each step asks: how many tokens may each request process?

Article 05 traces token budgets, RUNNING-before-WAITING, Chunked Prefill, prefix-aware admission & KV-pressure preemption.

https://t.co/pDa46d4xLT

#vLLM

[halfrost @halffrost]

vLLM’s heartbeat is EngineCore.step():

schedule → execute → abort barrier → commit

Article 04 traces request states, overlapped batches, optimistic counters, aborts, cleanup & continuous batching—one tick, line by line.

https://t.co/H2YNOohbK3

#vLLM

[halfrost @halffrost]

vLLM V1 is not “an API server calling a GPU.”

It is a process topology:
API server → ZMQ → EngineCore → shared-memory queues → GPU workers.

Article 03 traces GIL isolation, TP/DP math, startup barriers, and worker-death propagation.

https://t.co/QP0u7Fhavp

#vLLM

[halfrost @halffrost]

vLLM has 3 front doors: LLM.generate(), vllm serve, and OpenAI-compatible HTTP.

Follow the source and they converge:
process_inputs → EngineCoreRequest → EngineCore.

Article 02 traces offline, CLI, HTTP, SSE & cancellation.

https://t.co/MG3d72uN2z

#vLLM

[halfrost @halffrost]

What happens between LLM.generate() and the first token returned by vLLM? I traced the complete V1 request path—from input processing and continuous-batch scheduling to GPU execution, sampling, and output processing.
Part 1 of a source-code walkthrough:
https://t.co/8WIwBDJCcO

[halfrost @halffrost]

Do you remember when you joined X? I do! #MyXAnniversary

[halfrost @halffrost]

@TTTTTTTcp @luoleiorg 一起学习，一起进步，与君共勉🤝

[halfrost @halffrost]

@TTTTTTTcp @luoleiorg 哈哈，也没改命，还是菜鸟，还是屁民。

[halfrost @halffrost]

@Junlong_x @luoleiorg 这两年 AI 太火，最近我也在做一些关于 AI 的 research，blog 有一段时间没更新了。等作出成果以后，再出来写点文章。

[halfrost @halffrost]

@luoleiorg 祝福罗磊老师新年里心想事成！一切皆顺！

[halfrost @halffrost]

@luoleiorg 加油，祝福成绩 4 个 8

[halfrost @halffrost]

@ncubrian @iOSChenYilong 还没有，还在等 team match。今年的流程非常 slow。可能等到最后就没 hc 了

[halfrost @halffrost]

Do you remember when you joined Twitter? I do! #MyTwitterAnniversary

[halfrost @halffrost]

@lexrus 恭喜啊，7不容易

[halfrost @halffrost]

@luoleiorg 居然还活着。这几年大风大浪还能存活下来不容易。

[halfrost @halffrost]

@ncubrian 修复好了。抱歉我在换一个新的加密密钥，结果出现了这个问题。现在恢复了。感谢你的提��。

[halfrost @halffrost]

My GitHub Sponsors profile is live! You can sponsor me to support my open source work 💖 https://t.co/X4GuV7rppL