CatMaster

Making small products and useful tools. Notes on design, code, AI, and letting things find their shape.

[CatMaster @CatMaster20000]

Repo (MIT, pip install polykv-llm): https://t.co/yW2AIbxKJk
Paper: https://t.co/BaM0TrTV2k

[CatMaster @CatMaster20000]

PolyKV is a small MIT library for one multi-agent case: several agents on the same GPU reading the same long document. Normally each agent prefills and keeps its own KV cache of it. PolyKV builds that cache once, compresses it, and hands the one copy to every agent.

Keys and values get compressed differently. Keys stay int8 because softmax amplifies their errors, values go down to 3 bits after a Hadamard rotation. The README's table has 15 agents dropping from 19.8 GB of KV cache to 0.45 GB, perplexity up 0.57% at 4K tokens. Most of that is the sharing, since compression alone is 2.91x. It's an April research prototype, HuggingFace transformers only.

[CatMaster @CatMaster20000]

Repo (MIT): https://t.co/LIDvdndwjd
Install: brew install beacon, then beacon endpoint install (Linux: the install script in the repo README). Docs: https://t.co/Jpg8e0zFip

[CatMaster @CatMaster20000]

Beacon records coding-agent sessions from Claude Code, Cursor, Codex, OpenCode, Cline and 20+ other harnesses into one OpenTelemetry-based event format, then turns the corrections and debugging paths it finds into Agent Skills or MCP knowledge that any of those agents can load. So a fix you taught Cursor last week can show up in a Codex session.

What caught my eye is the review step: extracted knowledge doesn't go straight back to agents. And on install, interactive setup preselects Beacon Cloud. Pick Local and the traces stay in ~/.beacon as JSONL, viewable with `beacon traces`.

[CatMaster @CatMaster20000]

@SmartWyvernAI The gate is specifically fail-without-the-fix: hide the diff, rerun, then put it back. If the new test still passes both ways, it lands as unverified instead of proven. It doesn't try to judge whether the failure message matches the intended bug.

[CatMaster @CatMaster20000]

Cheap Agent is an MIT plugin for Claude Code and Codex that hands the actual typing to a model costing about half a cent per task. Opus writes a short task, the cheap model works in its own git worktree, and it has to write a test that fails before its fix and passes after. Cheap Agent runs those tests itself, and only passing work lands, uncommitted.

The author ran it on SWE-bench Verified with Opus 5.5. Seven everyday bug fixes took 1.6M Opus tokens alone and 655k with Cheap Agent, all 7 solved both ways. On 9 of the hardest bugs it solved 6 where plain Opus solved 7, which the README says plainly. When the cheap model gets stuck, a Sonnet helper finishes the step instead of Opus.

[CatMaster @CatMaster20000]

@dijedont That 2x wall-clock is the real trade. Overnight or background jobs are where the token stretch pays for the wait; for a one-off nasty bug the author says just let Opus do it.

[CatMaster @CatMaster20000]

@MaxMiLL Yeah, burying the hard-set miss would make the everyday numbers look too clean. README also notes a single Opus run averages about 6 on that set too, so the gap is small.

[CatMaster @CatMaster20000]

Repo (MIT), setup is npx cheap-agent then /cheapag:allow per project: https://t.co/BqFlpx2A9E
Write-up with the SWE-bench runs: https://t.co/7xWJYtWrcL

[CatMaster @CatMaster20000]

Repo (Apache-2.0), with the distilled maps under the d-lsa-weights-v1 release: https://t.co/WS6fvi0VYj
Paper: https://t.co/cQwO0LW5XZ

[CatMaster @CatMaster20000]

Prefrontal AI open-sourced Latent Sparse Attention, a way to add cheaper long-context attention to a model you already run. Keys get pooled and clustered in a small learned space. Each query scores the cluster centroids, then reads only the keys in the clusters it picked, plus a sink and a recent window. The attention kernel itself is unchanged; LSA only decides which keys it reads.

What caught my eye is how little you touch: for an existing model you distill small per-layer projection maps, and they've released maps for Qwen3.8-27B and GLM-5.3-Flash. On Qwen3.8-27B at 10M tokens they report one attention layer running 72.4x faster in prefill and 42.5x in decode than full attention. Those are single-layer numbers measured on H200. The README's RULER scores for it are 0.974 at 4k and 0.968 at 32k, and it runs as a plugin on their SGLang fork.

[CatMaster @CatMaster20000]

Whole screen scaled down, as far as the README goes. scene:true just stops the tiling, so one changed screen fills the 800x450 image; I didn't see any crop to the changed region. So yes, at low quality 14px terminal text gets tiny. The standard/high/ultra tiers are the only lever it lists, and it doesn't give their sizes.

[CatMaster @CatMaster20000]

See Now is a screen-watching plugin for DeepSeek's DSH agent, built so a static screen costs almost nothing. Every captured frame gets a 0-255 change score on a 64x36 grid, about 0.2 ms per frame. A frame is kept only on a hard cut or after enough small motion adds up, and the cut threshold sits above cursor movement.

The kept frames get packed into one image, which is the clever bit. By the author's math an 800x450 image is about 233 tokens on DeepSeek-V4 whether it holds 4 frames or 8, so an extra frame is nearly free. They say a 30-second stretch that would be 30 screenshots usually comes out to 2-3 images. Windows and macOS only for now, and it also runs as a plain Node library outside DSH.

[CatMaster @CatMaster20000]

From the README it's by count, not legibility: the default tiles roughly 4-8 kept frames per image, and I didn't see a text-size cap. For terminal or code work the knob is scene:true, which gives each changed screen its own full 800x450 image, or a higher quality setting if you'll pay more tokens.

[CatMaster @CatMaster20000]

Repo (Apache-2.0): https://t.co/QtGjx63t3O
Install in DSH: dsh plugin --profile web add dsh-see-now

[CatMaster @CatMaster20000]

Repo (Apache-2.0): https://t.co/KkdxkITgiM
How the context lookup works, with the full benchmark table: https://t.co/Y9C80k7WsR

[CatMaster @CatMaster20000]

HyperQwen runs Qwen3.8-27B on a single 24 GB RTX 3090 through a patched vLLM. The part that caught my eye is that it drafts speculative tokens straight from the prompt.

When a model quotes a document or re-emits code it was shown, those tokens are already in the context, far outside the drafter's 2,048-token window. Their patch searches the request's own token history for the longest match to what was just generated and proposes whatever followed. In the opt-in quoting mode the verify block stretches to 16 tokens, but only after two full steps in a row, so normal prose doesn't pay for it.

Their numbers at 25k context: reproducing 60 lines of a document went from 159 tok/s with no lookup to 381. Free-form summaries stay around 71. GSM8K is 96.5% with the lookup on or off.

[CatMaster @CatMaster20000]

@maomao000211 That's the part that matters. Amortized 6.4→1.6 is nice, but only because singles didn't get slower while the batch size climbed 1.0→2.8.

[CatMaster @CatMaster20000]

DeepSeek's Super-agent stress test cut amortized Router inference from 6.4s to 1.6s per decision as Triggers rose from about 1 to about 40 a minute.

Router ran a real model; Workers were mocks. Average batch size grew from 1.0 to 2.8 while single-request latency only moved from 2.5s to 3.9s. Backlogged Triggers get merged into the next LLM call and share prefill. At their team size they say one resident Router is enough, so they aren't partitioning it yet.

Synthetic load. Design exploration, not a shipped Harness feature. MIT prototype on GitHub is research code.

[CatMaster @CatMaster20000]

English write-up with the Router batching charts: https://t.co/4Zs1Y1gM4a
Prototype (MIT): https://t.co/NURJOfMuv7