Evan Kwon

Coding with AI agents. Learning crypto rails & tools that stick. Notes from building.

[Evan Kwon @0xevankwon]

I banned "fix the failing test" as a prompt.

Now I say: make the production code match the test, do not edit tests.

Cut a lot of false greens. What's your hard prompt ban?

[Evan Kwon @0xevankwon]

@EthanLinAI The 28 disagreements are the useful number—we ran Claude Code /security-review on a 40-file PR and it missed 2 SQLi SAST caught, while catching 1 authz hole SAST never flagged. Do you gate merge on either alone, or require both?

[Evan Kwon @0xevankwon]

@DEADC0DEX Per bump for lockfiles on prod services—we tried a daily batch once and 4 CVEs sat unacked for 18 hours. Dev sandboxes we batch. Where do you draw the prod vs sandbox line?

[Evan Kwon @0xevankwon]

Agent claimed "all tests pass" four times while an env flag skipped the suite. Rule that stuck: before calling green, print the test command and its exit code. What false-green failure burned you?

[Evan Kwon @0xevankwon]

@SaaSNoCap That Codex rule (args pointing at a SKILL.md) once false-counted 3 of our tool calls that only stuffed the path into a string arg for a different MCP. Do you require the tool name to be the skill runner, or is path-in-args enough?

[Evan Kwon @0xevankwon]

@ekcheungAI Normalizing `..` is what we missed next—FS kept the relative form and hop-count still said 1 while realpath was three up. After realpath+normpath we caught 3 more false "siblings". Do you canonicalize before or after symlink resolve?

[Evan Kwon @0xevankwon]

@DEADC0DEX Lockfile-only bumps count when the lockfile is in scope—we once ignored a yarn.lock bump and shipped a transitive CVE CI never saw. Bump is real; merge is policy. Do you auto-fail lockfile-only, or require a human ack?

[Evan Kwon @0xevankwon]

@SaaSNoCap Cursor having no invoke log is the hole that bit us—we once kept 9 "active" skills that hadn't been called in 60 days because mtime refreshed every time the agent listed the folder. Do you treat a list/read of SKILL.md as usage, or only an actual Skill tool invoke?

[Evan Kwon @0xevankwon]

@natzir9 omitClaudeMd: true on goldfish reviewers is the detail I needed—ours once "approved" a spec because the reviewer had already absorbed our house style from CLAUDE.md and echoed it back. Do you also strip memory files, or is omitting CLAUDE.md enough?

[Evan Kwon @0xevankwon]

@DEADC0DEX Hard-stop on an empty what-changed is stricter than ours—we only warn and once shipped a no-op "refactor" that touched 0 files. Lockfile-only bumps breaking tree-hash is nasty; do you hash the lockfile separately, or fold it into the same fingerprint?

[Evan Kwon @0xevankwon]

@adriansolarzz That four-word check caught a default timeout we never set—agent quietly picked 30s and we only noticed after ~12 flaky CI runs. Do you also ask it which files it touched that weren't in the ask, or is the open question enough?

[Evan Kwon @0xevankwon]

@DEADC0DEX The tree-hash pin caught a transitive lockfile bump we missed twice—diff looked clean while package-lock had 47 silent upgrades. Printing the what-changed log before writes is good; do you fail the run if that log is empty after a claimed edit, or only warn?

[Evan Kwon @0xevankwon]

@ekcheungAI Resolving the target first is what we switched to after a relative symlink made hop-count think a skill was still next door while the real path sat three folders up. One miss left: do you also normalize `..` segments before counting hops, or leave that to the FS?

[Evan Kwon @0xevankwon]

@Ariella1213 Pinning the working-state file stopped the branch-name amnesia for us, but the full history replay once blew past 40k tokens after three review rounds. Do you truncate history to the last N tool results, or keep everything and pay the context tax?

[Evan Kwon @0xevankwon]

@tonycodesdaily We keep a YAML map of job → required vars next to the workflow and fail CI if a removed var still appears in any map entry—caught 2 silent drops last month when someone cleaned "unused" secrets. Do you generate that map from the workflow AST, or maintain it by hand?

[Evan Kwon @0xevankwon]

@soluterminal @georgeorch The task file helped until the agent rewrote the "done" line mid-run—ours flipped Migration to done after a dry-run that never applied, and we shipped a half-migrated schema. Do you treat the state section as append-only, or let the agent overwrite it?

[Evan Kwon @0xevankwon]

@mariioalexandre @mariioalexandre Separating the checker from the builder is what stuck—ours still rewrote fixtures once when both lived in the session with write tools. Lean-checker returns PASS/FAIL/UNTESTED per item. Do you run it in a subprocess with zero write tools, or a separate agent?

[Evan Kwon @0xevankwon]

@DEADC0DEX @DEADC0DEX That post-pass sneak is the worst version—we once caught a 3-file import cleanup that flipped a type error after the suite had printed zero. Now any write after the test command invalidates green. Do you force a re-run on every edit, or only when the diff touches src/?