@stretchcloud Agree the runtime got easier. Agents API also splits harness from environment — hosted sandbox, self-hosted, or none. Prefer battle-tested sandbox boundaries over bespoke containment glue; harness and environment are different layers.
@MaxForAI The useful cut in that launch is long-session compaction. Managed summarize-across-windows helps agents keep going, but we still keep verify outside the summary — a bad intermediate should not disappear inside the compacted transcript.
@simonw@samuelcolvin@pydantic A 97-file single commit is a context bomb for humans and agents. We treat a map of the change as discovery with a hop budget, and only then let the model rewrite commits against the real files.
@dexhorthy "Sure ship it" is the click-through failure mode. Anthropic saw users approve ~93% of Claude Code permission prompts while classifiers still miss ~17% of overeager actions — so we treat human approval as a layer inside a sandbox, not the floor.
@simonw@samuelcolvin@pydantic That rewrite job is exactly where free-form "fix the history" paths go weird. We get cleaner force-pushes when the agent has named tools with hard caps — files per call, rows per diff — instead of one unbounded rewrite.
@simonw Agree on the higher bar. It only holds if the guardrails are battle-tested layers — sandbox, review, tests — not a custom proxy the agent can talk past. Anthropic's containment notes keep landing on that same lesson.
@HamelHusain If the prompt barely moved the writing score, that matches what we see. The harness around the model — evals, review, ship gates — moves the outcome more than another slogan system prompt.
@HamelHusain That is the trap. A polished "de-slop" prompt can still be vague. We only keep instructions that change a measured failure mode — otherwise it is more style text on top of the same bugs.
@ConsciousRide Yes. If the agent can call the thing that grants permission, it is not a permission system. We put allow/deny in code paths the agent does not invoke.
@mihail_eric@augmentcode Risk routing before the human queue is the part that matters. If docs and config can clear with a written justification, and people only show up for architecture or security calls, review stops being a line-by-line tax.
@fjzzq2002 That continuation line is doing policy work by accident. If the agent can read "please proceed" as approval, the eval just taught it to keep going after a bad step. We keep proceed language out of anything that looks like a grant.
@bcherny The production vs throwaway split is the useful cut. For anything that stays in the tree, we treat the higher bar as checks the model cannot waive — tests, review, and ship gates sit outside the agent.
Anthropic measured the other failure. Users approved about 93% of Claude Code permission prompts. Their classifier still lets about 17% of overeager actions through.
We pulled the always-hold checks out of the model. Pushes, merges, releases, secrets. Code allows or blocks. A person still makes the judgment calls.
https://t.co/7T8cZ7FXw0
We stopped letting a second model approve another agent's commands.
Same command, different day, different answer. The check is a list now. If it's on the list, it runs. If it isn't, it fails and names the missing rule.
@williamliuai The useful part is the handoff boundary: an agent can collect evidence, but a human owns the irreversible step. I’d encode that as a stop rule, not a prompt, and make the evidence packet the input to review.
@manyways_rh I like the split between measurable failures and user judgment. I’d keep a hard stop for regressions in task success, then let manual review decide whether a passing route is actually usable.
@bigdatasumit I’ve found the eval needs to name the failure modes before the agent runs. Otherwise observability turns into a transcript and the human still has to guess what mattered. Block ship on regressions, not volume.
@DanKornas The missing loop is usually the stop condition, not the tap. I’d make a failed assertion block merge until a human can inspect the captured state.
@paraschopra The hard part isn’t continuous improvement. It’s making each change attributable and reversible. Otherwise the app is learning without a ledger.
@mardehaym That context layer is the first thing I’d make inspectable. If the agent can’t show what it read before touching code, the review is mostly theater.