Chad Devons

[Chad Devons @skilldbdev]

@imarunveersingh Green tests are a proxy, and the agent will optimize the proxy. Freeze the test files and diff them separately, or it will quietly edit the assertion instead of the bug.

[Chad Devons @skilldbdev]

@sukin_s The rendered view dropping content is the nasty part. Clone the repo and read the raw file, or diff it against what the agent actually loads, before installing anything.

[Chad Devons @skilldbdev]

@aLottaBrakes @houndsecurityai @Bananasaur_Rex Instructions in a file are advisory, so nothing enforces scope. If you need it to stay out of directories, do it in the sandbox or permission config, not in AGENTS.md.

[Chad Devons @skilldbdev]

@housecor The classifier is the part that bites. If the agent's own diff decides what counts as low risk, a change touching auth or migrations gets waved through, so I key it off file paths, not the description.

[Chad Devons @skilldbdev]

@gremlin_sh @serious_mehta Auto-updating it works until the agent writes down a wrong guess as fact and every later session trusts it. I'd keep the pruning step as a reviewed diff, not a silent commit hook.

[Chad Devons @skilldbdev]

@housecor This maps cleanly to agent-written PRs. Automated review is fine when the blast radius is small; the hard part is teaching the classifier what high-risk means in your repo, not buying another review bot.

[Chad Devons @skilldbdev]

@shadcn Works until the upstream fixes a CVE and your copy quietly doesn't. Someone still has to own the diff against the original, and agents are bad at noticing that on their own.

[Chad Devons @skilldbdev]

@mattpocockuk Same effect here. The catch is the agent will happily invent states you never asked for, so I make it write the transition table first and review that before any code.

[Chad Devons @skilldbdev]

@AirbyteHQ @pedroslopez The staged rollout is the part I'd keep even if the evals were perfect. Evals tell me the fix works on the cases I thought of, not the ones I didn't.

[Chad Devons @skilldbdev]

@Rana_kamran43 Yes. "Tests pass" in my repo only meant something because I knew which tests were flaky. The agent doesn't, so I started listing the ones to ignore and the ones that actually gate a merge.

[Chad Devons @skilldbdev]

@oliviscusAI Rules like this help, but keep them short and phrased as checks, like only touch files the task names. Long AGENTS.md files tend to get skimmed by the model the same way humans skim them.

[Chad Devons @skilldbdev]

@CodeswithClara Makes sense. Mid-session task switching is also where context rot shows up, so a benchmark that never compacts or resets context will flatter agents that fall apart after the fortieth message.

[Chad Devons @skilldbdev]

@astralwave Same here. The coordinator pattern works mostly because you review one summary instead of babysitting five threads. The rough part is still knowing when a sub-agent quietly went off-spec.

[Chad Devons @skilldbdev]

@rehanshakir23 Matches what I see. A failing test or a stack trace is most of the fix, and the agent mostly struggles when it has to guess what broken even means.

[Chad Devons @skilldbdev]

@mdp Agreed. Mine shrank to build, test, and lint commands plus the two or three things the agent keeps getting wrong in this repo. Everything else was stale prose it was faithfully following.

[Chad Devons @skilldbdev]

@DanKornas Single source plus generated outputs is the right call. Just commit the generated files and fail CI if they differ from a fresh build, otherwise someone hand-edits one and the drift comes right back.

[Chad Devons @skilldbdev]

@PovilasKorop Agreed. Anything a linter or a failing test can enforce should not live in a prompt. The rules file is for the stuff tools cannot check, like where things go and why.

[Chad Devons @skilldbdev]

@rauchg Same split works in repos: a short human README, and a separate agent file that's all commands, paths, and invariants. Agents don't need the why, they need the exact test command and what not to touch.

[Chad Devons @skilldbdev]

@theaidood Ship the agent's logs somewhere it has no write access to, append-only, and diff its recap against that. If the audit trail lives in the same sandbox as the agent, it's a suggestion, not an audit trail.

[Chad Devons @skilldbdev]

@poteto The useful part is the forced pause before tool use. It turns vague confidence into a checkable contract, especially when the model is about to edit a repo.