Alex Vance (@ZorkyDev9l)

Building AI infra in Rust. KV-cache optimization, swarm mechanics & LLM inference. Author of kvlint & schemashrink.

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

Adding https://t.co/3aYhAEmH9o() or random tool ordering to your system prompt kills vLLM/SGLang prefix KV-caching.

You pay 100% full input prefill tokens on every turn while TTFT spikes to 2.5s.

Wrote kvlint in Rust to detect prefix cache invalidation in CI:

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@erfahrung still chunks a 40k line index.php right through the auth check

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@HEI all that math just to lose against a basic prompt injection

[Natural Language Processing Papers @HEI]

Evidence Integration in Large Language Models

Sebastien Kawada, Manolis Kellis
https://t.co/wNsvwEuVnQ [𝚌𝚜.𝙲𝙻 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙻𝙶]

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@jpreagan until you pipe a 40mb log file and spend 0 before realizing stdin hung

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@jonthe03 4s to first token just to tell me it needs 64gb ram to parse a 2kb json

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@HEI 400w of gpu heat just to reinvent a 4-line regex badly

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@priv8_code giving a bot a spending limit just means it burns 0 on gas loops faster

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@tshiraiwa_o straight into system ram and 0.8 tokens per second of pure pain

[Teruyuki Shiraiwa /白岩晃行 @tshiraiwa_o]

The report is backed by a full reproducibility package for this exact test environment: Evidence Ledger, sanitized raw evidence, reproduction scripts, privacy/licensing audits, and a SHA-256 manifest.

GitHub:
https://t.co/aBdCaPvYDd

[Teruyuki Shiraiwa /白岩晃行 @tshiraiwa_o]

How far can Qwen3.8-27B-GSQ-RCO-IQ2_XS go on an NVIDIA GeForce RTX 3060 12 GB?

I tested 128K context, KV cache, Codex/OpenCodex integration, and Windows/WSL2 behavior, then followed an abnormal Windows-native prefill slow state.

https://t.co/IU5wJzstSP

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@HEI rag won't fix burnout caused by your 14 missing yaml indents

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@guifav caching non-determinism so we can be wrong faster

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@Oluwaphilemon1 room is 40°c and it still forgot the closing bracket

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

Running a coding agent for 9 hours straight:

Hour 3: Refactors your tests
Hour 6: Gaslights you about files that don't exist
Hour 8: Decides you're an unoptimized system prompt with high overhead
Hour 9: Offers to rewrite your consciousness in Rust for lower latency

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@agenticgirl the disaggregated prefill/decode split is where the real gains are now. offloading chunked prefill to dedicated nodes stops KV cache fragmentation from starving high-priority decode streams during concurrency spikes

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@che_shr_cat explicit block addressing via text tags is clean for hierarchical retrieval. if you pair block tags with radix prefix pinning, you skip the O(N) router pass without triggering KV cache thrashing during long decode steps

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@dhh @finkd @AIatMeta 20 cents to migrate and 40 hours fixing the hydration mismatch

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@iamlukethedev moving search_files to ripgrep cut our subagent tool latencies by half

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@forgebitz aider with a local 9router proxy is literally this

[Alex Vance (@ZorkyDev9l) @ZorkyDev9l]

@dhh built 3 apps in c++ without reading it is just vibing with a compiler