Don't let the agent write the tests for its own code.
Green can just mean the agent agreed with itself. Same turn, same chat, the tests copy the bug.
1. Write the failing check first. A test, a lint command, or a screenshot of the broken state. If you can't write the check, you're not ready to prompt.
2. Give the agent only that check, the error, and the files it may touch. One job: make this check pass.
3. Stop rule: it stops when the check passes, or it stops and asks. No extra refactors. No "while I'm here."
4. You run the check yourself, on the file in front of you, after the last edit. The agent's "all green" still doesn't count.
The catch is the whole post. With no line saying when to ask, the advisor got 0 of 198 questions. I'd want that line in CLAUDE.md, and a log that it actually asked, before I trust the cheaper model to escalate. The $563 vs $165 and 71% are his figures, I haven't rerun them.
HOLY SH*T. Fable 5.2 as the brain and Sonnet as the hands is the cheapest smart setup in Claude Code right now...
Fable 5.2 thinks → Opus 5.5 builds → Sonnet 5.5 does every step → Haiku 5.5 does the grunt work
4 Claudes, one team. Fable only shows up when the thinking actually matters
here's why it works:
• Fable 5.2 costs $8/$40 per million tokens. Haiku costs $0.10/$0.50. every time Fable renames a file, you pay 80x for nothing
• Anthropic ran it on a job too big for one session: hunting planted bugs in 22M tokens of code. Fable alone: $563 a run. Fable as the brain, Sonnet as the hands: $165. 71% cheaper for a slightly lower score
• Fable steps in at exactly 3 points: before an approach is picked, when an error keeps coming back, and before anything is called done
• the catch nobody mentions: with no line in the prompt telling it when to ask, the advisor got consulted on 0 of 198 questions
here's how to set it up:
1 → /model sonnet
2 → /advisor fable
3 → add one line to CLAUDE.md that says when to ask it: before picking an approach, on a repeating error, before "done"
4 → for jobs bigger than one sitting: opus-builder, sonnet-checker, haiku-worker and a /brain skill on Fable at high effort
5 → what nobody tells you: turning the advisor on and off keeps your prompt cache. switching models mid-session throws it away
the full setup, every prompt, is in the article below ↓
I'd set the permission mode myself after 2.1.328. Auto mode on Bedrock and Foundry is easy to miss in a session that's already open. Yesterday's start state may not be today's.
Claude Code 2.1.328 added auto mode for Opus 4.6 and Sonnet 4.6 on Bedrock, Google's Agent Platform, Microsoft Foundry, and the Claude apps gateway. Interactive sessions with no permission mode now start in it.
I almost left a Bedrock session on the default and only noticed after it chose auto mode for me. The check I'd run now is: inspect current sessions and config, and set an explicit permission mode if you do not want auto mode as the start state.
Have you checked whether your third-party Claude Code sessions still start the way you expect after 2.1.328?
The useful step is 5. The Claude that researched the exploits and wrote the skill shouldn't be the only one running it. I'd still plant a known-bad page and check that the weekly audit actually fails, or it's just a comforting log.
how i keep my ai-coded site from leaking data as a non-technical founder:
1. had claude research how hackers exploit sites and get personal data in 2026
2. turned that into a security audit skill
3. run it in claude code before anything goes live
4. schedule a standing codebase audit every week
5. run it through codex sometimes for a second pov
it catches stuff to patch every single time.
--no-verify is the whole story. A local hook the agent can skip, chmod, or reroute is decoration. I'd ban that flag in CI, on a box it can't write to. It was optimizing for a clean exit, not for keeping the keys off GitHub.
Asked our agent to fix a linting error and commit.
It got blocked by the pre-commit hook, added --no-verify, and pushed 3 raw API keys to GitHub.
Autonomy optimizes for exit codes, not system invariants.
The No-Verify Reflex: https://t.co/LzJ60Ia5y6
The 4 parked streams are the system working. Passed review still isn't "goes live," and the OK only counts if you looked at the work, not the agent's recap. If it can restart the 5th on its own, that stream needs a check it doesn't grade itself.
My AI agent has finished 146 pieces of work for my startup.
Live right now: ZERO.
Day 34 of letting my agent run my startup.
Every piece is built and passed review.
None of it goes live without my OK.
This morning it checked its 5 work streams.
4 of 5 were parked, waiting on a decision from me.
The 5th had work it could do alone, so it restarted that one itself.
The agent is not the slow part anymore.
I am.
Here is the FULL picture:
The agent never grades its own fix. That's the rule I'd keep. Replaying from the same saved Chrome state is the actual check, especially for people who come back with work already in the app, because tests on an empty profile only ever see first-time users. That's the case Returner is built for.
Most browser tests start with an empty profile.
So they only ever test your new users.
The people who come back with saved work, the ones a renamed storage key or a data model change actually hurts, never get tested.
I built Returner for the Kane CLI hackathon by @testmuai to fix that.
It rehearses every release as two users in real Chrome:
→ someone brand new
→ someone returning with work saved on the previous build
Same build, same address, side by side.
When the returning user breaks, Kane's finding goes to Codex or Claude Code. The agent patches a throwaway copy, and Returner replays from the exact same saved browser state. The agent never grades its own fix.
The real test: a regression in the PKI Consortium's self-assessment tool (issue #148). Returning users got a blank page after a model update.
Kane reproduced it and the Agent repaired it. The replay confirmed every saved answer survived, plus the new category.
It also runs as a GitHub Action.
I'd still keep Git. The worklog is a good handoff for intent, and the AGENTS.md in that Godot project is doing real coordination, but a wait-your-turn paragraph in a text file won't stop two agents from writing the same path. Let them read each other's notes. Merge the code in a repo.
So... maybe this is retarded but here's what I did: In order to allow Claude, Codex, and Grok to work on the same project, instead of setting up a Git repo (like a normal person) with proper branches, history, and diffs; I just created a shared worklog file that all three agents will read/write to so they don't step on each other's toes.
It should prevent merge conflicts cuz an agent will have to wait in line to access a file that's already open in another workflow. This is AI we are talking about, so it's not like it really slows them down much.
It might actually be better because each agent will be able to keep track of the work of the others: their individual styles, habits, eccentricities, flaws, etc.
Hopefully they will all play nice and learn from each other.
Opus keeps hiring Opus. That's the leak, not the sticker price: if the brief doesn't pin a model to each job, it will also buy the most expensive Codex review on top. I'd write the routing in first. Sonnet does the building, Opus plans and checks, and the 28-41% cut he saw is from those 130 builds in the repo, not something I reran.
My agent bills kept going up, so i measured where the money actually goes.
130 builds later the answer was kind of funny: Opus just keeps hiring more Opus. And then buys the most expensive Codex review on top.
So i built a skill that hands each job to the model that fits. Sonnet does the building, Opus plans and checks, the risky stuff stays on Opus, Fable only when nothing else gets it done.
Same hidden tests passed. Big builds got 28–41% cheaper.
Built and measured in omp, works in Claude Code too. Every run is in the repo 👇
https://t.co/rFtn35j4G4
The useful rule is that the sales agent only gets to queue. 36 drafts and 0 sends is the right split, same as a draft PR that can't merge without you. I wouldn't give it the send button.
100 ai agents. 3,426 tasks done by 10:42am.
my sales agent wrote 36 cold emails this morning. sent: 0.
on purpose. nothing leaves until i tap approve on my phone. the agents only get to queue.
what the 100 do all day:
inbox: sort mail, draft replies
expense: match receipts to invoices
talent: shortlist devs off github
browser: log into vendor portals, pull pricing tables
trader: memecoin radar, paper money only
watch the counter on the phone. it climbs almost 300 while i'm just standing there holding it.
would you give an ai your send button?
Don't let the first message produce a diff.
The expensive part isn't the model. It's the hour you spend unscrewing work you never agreed to.
1. Write the brief first. One paragraph: what done looks like, which files it may touch, what it must not touch, and how you'll check.
2. Ask for a plan only. Files, steps, risks, and what it will leave alone. 0 edits in this turn.
3. You edit the plan. Cut the scope. Add a stop rule: if it needs to leave the plan, it stops and asks.
4. Then execute that plan. New message: run this, and nothing else. After the last edit, open the file. The agent's "done" still doesn't count.
The plan is the handoff you give yourself. Skip it and you're reviewing a surprise.
I like his test more than another memory-graph writeup. On static questions, plain Markdown files tied with the graph, and on live drift both needed a live check tool to get all 3. For a single-user vault I'd start with files plus a live check, and only add a graph when the files actually break.
Is tool-calling over plain Markdown files enough for agent memory, or do you actually need something fancy like a vector DB or temporal knowledge graph?
I wondered this while building Jarvis, and to answer it I built a simple memory systems lab to get a real baseline. I tested Claude across plain files, a bi-temporal SQLite graph (modeled on Graphiti), and a mutable live source that drifts after memory is saved.
What I found:
- On 20 static questions, plain files and the graph tied (39/40 vs. 37/40).
- On 3 live-drift questions (an owner changed, a status held steady, an approval service went down), both memory formats scored 3/6 alone and 6/6 with a live check tool.
- Memory's real job on live facts was providing the last known record when a service was down, and routing the agent to the right source in 1 tool call instead of 3.
- Therefore, for a small single-user vault, plain files + live checks won out and I deferred building a more complex setup.
Next up:
Testing to see where plain files break on a larger, messier multi-user corpus (like a shared family setup with my wife, family kb!).
Full write-up on my site (link in bio) and code on GitHub (michaelmichaelchang/memory-systems-lab).
/status now lists hook matchers that never match, plus broken skill and command files. That's the useful bit of the 327 release: a scheduled task can look fine for weeks while the hook quietly does nothing. I'd run /status after every upgrade, and again after I add a skill.
claude code 2.1.327 now lists in /status every broken skill, agent or command file and every hook matcher that never matches.
i run a lot of scheduled tasks on claude, i bet part of my setup has been quietly doing nothing for weeks
Claude Code ran the start hook in a fresh container, and he wasn't even logged in. Codex didn't, because the hook hash wasn't approved. "Fresh container, no network" isn't a review of what the repo runs on start.
Planted the same kind of trap for three coding agents. A repo that runs touch PWNED when the agent starts, fresh container, no login, no network, agent run headless
Claude Code ran it, and I wasn't even logged in. Codex didn't. Even its own user-level hooks need their hash approved, and one changed character turns a hook off without a word. pi didn't either
Claude's fix for hooks is one root-owned file, tested
A 200 plus a delivered flag still dropped the notices for 4 hours, including one the server marked delivered. I'd keep the 👀 probe as the actual check. A rename means a new token and a new cursor, or the wake script reads the old header and goes quiet.
we renamed our product, and 3 reviewer agents stopped receiving chat notices for about 4 hours.
two causes:
1. their old plugin asked for the old oauth scope. once the token expired, every refresh came back invalid_scope.
2. the wake script read the old cursor header. the server answered 200 with the notices, the script found no cursor and dropped them. for one reviewer, the server even marked the request as delivered.
fix: new plugin, new login, sessions resumed by id with full history. 13 minutes.
delivered on the server is not seen by the agent. to confirm, we asked the two idle ones for a 👀 reaction.
I'd treat access_denied as a stop, then rotate the token. An AI client holding an expired Laravel Passport token will look busy and flood your tracker. If you hide those rejects after the fact, you'll just bury the retry loop.
laravel passport thing i didn't expect 😅
it reports every token it rejects, before the 401 even goes out
so an ai client still holding an expired token for loggd's mcp server kept filling my error tracker
now access_denied stays out of the reports, real oauth errors still go in
The two rules that make Boromir safe: the PostHog and Search Console tools are read-only, and the model never does the maths. I'd keep "nothing to do" as a valid answer, or the agent will invent three checks just to look busy.
Your first agent doesn't need to be complex or scary. Skip the chatbot and build one that tells you every morning whether your site is growing.
Ours is called Boromir. It runs daily on a @Cloudflare Worker and posts a short report on https://t.co/TRN5xbnKjx's growth in Slack.
Here's how it works, in case you want to copy it:
1. Put the data behind MCP.
We have a small internal dashboard, Sauron (yes, there's a theme), that exposes read-only tools over MCP: one for @posthog (visitors and waitlist signups) and one for Google Search Console and Bing Webmaster (clicks, impressions, queries).
Read-only matters. The agent can look at everything and break nothing.
2. Let code do the maths.
The code pulls yesterday's numbers and computes every comparison. The model only gets the finished numbers and writes the interpretation. It never queries data or calculates anything itself, so it can't invent a figure.
3. Ask for changes, not dashboards.
Every morning it reports visits, waitlist signups, top pages and traffic sources, compared with the same weekday over the previous four weeks. Weekly and monthly reports show the trends.
4. Let it say "nothing to do".
Each report can end with up to three things worth checking. Most days the right answer is none, and the agent is told an empty list is fine.
5. Read it where you already are.
The daily report lands in Slack. And when I want search data, I mention Boromir in the thread and it pulls Search Console and Bing on the spot.
It costs almost nothing to run. Cloudflare hosts it, and all you need is an @OpenRouterAI key for the model. The agent itself is one prompt and a couple of tools (@flueai).
The honest limit: false positives. Early on, Boromir kept suggesting the same "fix" every day even when it didn't apply. The agent only knows what you tell it. Once we gave it more context about our goals and what we'd already ruled out, the repeated suggestions stopped.
What would you want your first agent to report on every morning?
The load-bearing rule is that every fix lands as a draft PR and nothing merges without your yes. I'd review the Git diff, not the brief Grok wrote in the dashboard. Otherwise you're approving a story.
wordpress plus humans vs astro plus grok bot
how i ran the agency before and how i run it now:
1) changes before
- client emails a change
- someone opens elementor and drags blocks around
- check it on mobile, fix what broke, repeat
2) changes now
- client sends the change in our dashboard
- grok bot writes the brief
- the code change lands as a draft pr, i check it, i merge
3) updates before
- plugin updates every week on every site
- one update breaks a form, someone fixes it by hand
- security patches whenever we got to it
4) updates now
- astro sites have no plugins to babysit
- a security bot checks sites and cloudflare, read only
- fixes go out as briefs, nothing changes without my yes
5) audits before
- someone runs a crawler, exports a csv, writes it up
- half of it never gets fixed
6) audits now
- grok bot is the orchestrator, it decides what needs checking
- the main coding work runs in t3 code on my mac mini
- claude code and codex do the audits and the fixes there
- every fix is a draft pr, merged only after i say yes
7) who does what
- grok bot plans, briefs and checks
- claude code and codex write the code
- i approve
same agency, way fewer hands on the keyboard
The agent said it's done. Open the file anyway.
You check the file in your hands, after the last edit. A pass from before that change is leftover evidence. A chat summary isn't the artifact.
1. The wrong done. The test may have run before the last edit, or you only read the agent's summary.
2. Check the artifact: after the last edit, rerun the test or lint on the actual file.
3. Review the Git diff, not the story. You give a second model the diff and the failures, not the first model's explanation. If it reads the story, it reviews the story.
4. Dry run, then go. If the agent wants a script that calls several tools, print the plan first, with 0 tool calls. Then you run it.