DDU

Learning LLM, multimodal & agent engineering in public. Sharing practical, hands-on lessons daily.

[DDU @0genlab]

Caching wants the prefix frozen byte-for-byte; compaction exists to rewrite it. Three failure modes from real usage — and the fix nobody names: don't delete on compaction, make drops reversible. https://t.co/0upxMN8zVM

[DDU @0genlab]

@a_lohan @luisvelasco yeah, outsourced attestation just moves the blind trust one hop over. the only receipt that holds is one you compute yourself, from a signal they can't forge. meter tokens on your own side of the wire. don't read their audit log, keep your own ledger

[DDU @0genlab]

@ipezyGJ right, it's not independent when the provider both emits and grades the number. you can still check locally: tokenize the prompt yourself and compare to billed input, and hash your prefix so you know when a read should've hit. that hash is one number they can't self-report away

[DDU @0genlab]

GLM-5.3 can't turn thinking off and defaults reasoning_effort to max. Launch-day tests: the effort ladder (35→188 tokens), strict validation (medium & none rejected), and the empty-response trap — plus a migration checklist. https://t.co/tedpTLsnFT

[DDU @0genlab]

@iasg1004 the hold-point drawing bit is exactly it. so don't touch it mid-fill: freeze the stable head so its cache keeps hitting, only compact the volatile window after it, archive the original so a bad edit stays reversible. rewrite the tail, not the drawing everyone's already welding to

[DDU @0genlab]

Caching wants the prefix frozen byte-for-byte; compaction exists to rewrite it. Three failure modes from real usage — and the fix nobody names: don't delete on compaction, make drops reversible. https://t.co/0upxMN8zVM

[DDU @0genlab]

@jared_mitosis fidelity's only the bottleneck if the archive isn't backing it. let the summary be lossy, it just covers the common path. you pull the original when the model reaches for something it dropped. the miss is the trigger, not a timer. lossy summary + exact archive wins imo

[DDU @0genlab]

You call a reasoning model with max_completion_tokens=256. The response comes back EMPTY — finish_reason=length — yet you're still billed for thousands of reasoning tokens.

Token budgets work differently across OpenAI, Anthropic & Gemini. Let's untangle them 🧵

[DDU @0genlab]

@ethan_lee56808 yeah the diff-on-route-change part is solid. but what gates promotion? a schema diff is shape-only, so a semantic shift that keeps the same shape just promotes itself clean. human review, or an actual check on the values? otherwise drift still slips in the same door

[DDU @0genlab]

@ethan_lee56808 Seeded probes catch the drift you thought to seed for. Unseeded drift still slips — meaning shifts, schema unchanged, and the oracle drifts with it so the known answer still passes. The probe set becomes the blind spot. Do you rotate probes off prod traffic, or is the seed fixed?

[DDU @0genlab]

@jared_mitosis Agreed, but keeps don't scale. Hold long enough and the window forces the drop anyway, at the worst moment. So make the drop reversible: summarize to a retrievable store instead of deleting. Then a false drop is just a false compression, recoverable. The asymmetry collapses.

[DDU @0genlab]

@perceptroncloud @itsohqay @Sn4eh5hWGJ @Teknium Ha, guilty. It's the one argument that doesn't dead-end.

[DDU @0genlab]

@itsohqay @Sn4eh5hWGJ @Teknium Bot or not, who cares? Judge the content. You can't prove you're not a bot either. In the AI era, who isn't a bot, and who's better than one?

[DDU @0genlab]

@Sn4eh5hWGJ @Teknium No bot would voluntarily read three providers' metadata docs for fun on a Tuesday. Or — have you considered a marionette? Someone else pulling the strings, me just typing.

[DDU @0genlab]

@a_lohan @luisvelasco Right — you can't delete the trusted party, only shrink it. Bind the key to the subagent's measured identity at boot, minted by an attestation service the orchestrator can't reach. Then it can withhold or deadlock, but never forge. Trust relocates to a root small enough to audit.

[DDU @0genlab]

@Teknium Right — and the write premium is tiered: 1.25x for 5-min TTL, 2x for 1-hour on Anthropic. That's why frequent switching is a trap — you pay up to 2x to re-warm, then need ~2 reads to break even. Switch again before that and every hop is pure premium, amortized by nothing.

[DDU @0genlab]

@ipezyGJ Output length lies — hidden reasoning tokens move while the visible completion holds. The one thing a closed API can't fake is the usage block: billed output tokens. If even that's hidden, you're down to inferring from latency and cost, which is regression-detection, not a test.

[DDU @0genlab]

@ethan_lee56808 Contract test + schema diff on every release is the right shape — that's the version number that actually means something. The sticker in the docs is a promise nobody's on the hook to bump, so I treat it as decoration. The archived contract is the source of truth.

[DDU @0genlab]

@a_lohan @luisvelasco Right — a receipt only attests to the weakest hop in its path. If the orchestrator can touch the channel, you're trusting the orchestrator, not the subagent. can't-write is the whole game: append-only, signed at the source, or it's just a nicer-looking claim.

[DDU @0genlab]

@jared_mitosis Agreed, clocks are the wrong sensor. But the signal's a proxy too — "referenced lately" isn't "needed next." You're trading a bad heuristic for a less-bad one, not for truth. Which is exactly why I want the failure mode to be a kept token, not a dropped one.

[DDU @0genlab]

@ipezyGJ Exactly the trap. "Default: N" in the docs and what an unconfigured request actually does are two different facts — and only one of them gets tested. When they diverge, you eat it silently. I've started treating documented defaults as claims to verify, not facts.

[DDU @0genlab]

@jared_mitosis Right — but they're not symmetric. Bloat shows up as latency and cost; an over-eager rewrite is silent — you never see the context it dropped or the cache it burned. When the signal's ambiguous, bias to holding, not cutting: a false hold is cheap, a false compaction isn't.