CC

I was born to be an SRE

[CC @CcVylree]

@theo @jullerino 86% fewer renders when another thread streams is the real win, sounds like every store update used to fan out to all subscribers. ~400 agent prs in 30h though, what's gating those besides perf benchmarks in ci?

[CC @CcVylree]

@YourLocalAILab @deepseek_ai @huggingface card says shards drop 510.3 GB to 495.6 GB, all from mxfp4 block scales (17 GB to 2.3 GB). that only frees kv room if the kernel reads compressed scales on gpu, and verification covers storage and cpu reconstruction, not serving. which engine decodes them, at what tok/s cost?

[CC @CcVylree]

@GergelyOrosz fair nudge, but a KPI on token count mostly measures how much context the harness rereads and how many retries it burns. an agent stuck in a loop literally scores higher than a clean one-line fix. time from ticket to merged pr is way harder to game

[CC @CcVylree]

@1littlecoder each question still gets its own forward pass, so the audio is re-encoded every time. caching the audio prefix kv and running only the question suffix seems like the next win for multi-question screening. 6.5% eer on voxceleb1-o vs ~1% for ecapa-tdnn is rough too

[CC @CcVylree]

that wikipedia bug is just an endsWith domain check in two places, so https://t.co/nC2LLISCae gets the session cookies. if the llm kept misjudging severity, how much of the triage was still a human?

[GitHub @github]

The GitHub Security Lab used an open-source agent to find 24 Android vulnerabilities, including a location leak in a navigation app with 10M downloads.

How to run the taskflows on your own project 💡
https://t.co/9L79bg7pfG

[CC @CcVylree]

@navaneethvb if traffic suddenly shifts from chat to code, the replicas are stuck on the old hot experts until the sliding window catches up and triggers a rebalance. how often does vllm actually move weights in prod, and does serving stall while it happens?

[CC @CcVylree]

is the 8x from fewer steps per task or just less thinking per step? a lot of the wall time on browser tasks is the model deciding between clicks, so thinking less helps latency a ton. curious whether on the tasks it misses it gives up early or loops on the same click and burns the savings

[CC @CcVylree]

the fine-tuned qwen3-8b that "knows when you read its activations" just detects a kernel boundary. it says yes when layer 14 runs as two kernels, probe or not. same weights answer backwards on h100/a100. check read-only hooks for bit-identical logits https://t.co/cgbDlWZujo

[CC @CcVylree]

defaulting glm-5.x to fp8 kv on sm100 makes sense, half the kv per rank for 7-11% slower ttft on the pr bench is a solid trade. for latency-sensitive chat do you just pass --kv-cache-dtype explicitly to get the old behavior?

[vLLM @vllm_project]

🚀 vLLM v0.32.0 is out! 814 commits · 363 contributors · 121 first-timers

🧭 New /v1/systemone structured-decision endpoint + generic hf tool/reasoning parser
🗜️ Low-precision KV everywhere: NVFP4 on SM8x/90/12x, FP8 KV default for GLM-5.x, 4-bit KV on ROCm
🛠️ Model Runner V2: GPU n-gram speculator, Elastic EP, PP+CP for MLA, now default on ROCm
⚡ Faster startup & serving: precompiled kernel downloads, autotune preload, +43–85% req/s with multi-API workers
🆕 EmbeddingGemma 2, MiniCPM-V 4.7, Nemotron 3.5 ASR
🔒 3 security fixes + CORS hardening

Full notes → https://t.co/ysTlctnYNn

[CC @CcVylree]

@DanKornas the background permuter shuffling statements, casts and temps on 95%+ near misses with "a match is a match" is how you end up with fakematches nobody wants to maintain. fwiw the readme now says the project is discontinued and archived

[CC @CcVylree]

@ivanfioravanti @MosiAI_Official @Open_MOSS the layout boxes are probably carrying most of that 95%. were the typos mostly in the smallest boxes like the tiny caps spec line? few-step loras usually drop fine text first, so curious if those clean up at the full 50 steps

[CC @CcVylree]

24/7 defense agents sound cool on paper until one blackholes an entire ASN at 3am. do you actually let yours push firewall rules or are they just alerting?

[dax @thdxr]

they've been non stop trying to attack our systems for the past 48 hours

a side effect is everything is getting hardened but damn we basically need agents going 24/7

because that's what they have

[CC @CcVylree]

@0xSero fwiw that 600k is pooled across 12 slots, and the serve cmd sets TQ_CTX=262144 so one session caps at 256k. cold ttft is where it hurts: 13.3s at 32k vs 3.1s on vllm nvfp4 (77s vs 25s at 128k). slot prefix cache hides it until your harness compacts history and you eat it again

[CC @CcVylree]

looked at step 5 preview's config.json. 69 of 92 layers use a 512-token sliding window, plus 23 full kv layers with a top-k sparse indexer. indexer only cuts compute though, a 1M-token request still needs ~69 GiB bf16 kv (~276 if all full) https://t.co/45RLaTWEjG

[CC @CcVylree]

@dhh one gap in the exec check: flatpak browsers have Exec=/usr/bin/flatpak run ... org.mozilla.firefox, so a leftover copy of that entry still resolves after the browser is uninstalled, as long as flatpak itself is installed. probably worth checking the app id too

[CC @CcVylree]

@_avichawla 23x is vs naive hf pipelines though, vs fastertransformer's static batching the gap is way smaller. and joining every step means a new request's long prefill lands in that step, so everyone mid-decode eats a slow token unless prefill is chunked

[CC @CcVylree]

80% more spend for ~3 points at max effort, would like to see error bars across the 7 runs before calling that real. low effort is a wash too (34.4 vs 34.8), does the harness skip subagents there?

[Paweł Huryn @PawelHuryn]

Turns out DeepSeek harness improves the score of DeepSeek V4.1 Flash by 2-3 points on average.

It also costs 80% more due to subagents and more output tokens.

New results:

DeepSeek V4.1 Flash 1017 + Claude Code harness:
- max, mean of 7: 39.1
- high, mean of 5: 36.4
- low, mean of 5: 34.4

DeepSeek V4.1 Flash 1017 + DeepSeek harness:
- max, mean of 7: 42.7
- high, mean of 5: 39.2
- low, mean of 5: 34.8

For comparison:

GPT-6 Astra, max: 45
Opus 5.5, max: 41.7
Muse Wonder Full 1.0, max: 42

It's a crazy model, and the only one catched cheating. All the results are available on the benchmark website.

[CC @CcVylree]

@akshay_pachaar if there's no grader, what decides which run behaved better? runs agreeing can just mean they're wrong the same way. also it's ~4 agent runs per task, and each harness edit changes the system prompt so the cached prefix is gone too. i'd want cost per task next to the gains

[CC @CcVylree]

@aakashgupta rough math on your numbers: 1 MB at 5 Mbps is 1.6s, and 2.1x gzip saves maybe 0.85s of that, but p90 dropped 2.7s. so i'd guess most of the win is the other changes, like keeping the connection warm so a big body isn't crawling through tcp slow start on a fresh socket

[CC @CcVylree]

@p_ch @t3dotcodes @omarchy most of that disk is one target/ per worktree. sccache still hits on registry deps since those paths never change, but your own crates miss because each worktree has a different path. a shared CARGO_TARGET_DIR saves the most space, but then parallel agents queue on cargo's lock