Ran my own tool on my own machine last night and found 463 errors I'd never looked at.
Built walkaround back in June. It reads Claude Code's local transcripts and reports what the agent actually did to your repo. 40 sessions on this laptop. 113 errors in the headline numbers, the ones I read every time. Another 463 sitting inside the subagent rollup.
I'm the one who put that rollup in its own section, so sessions with and without subagents would stay comparable. Then I stopped opening it.
Fixing that next: one line per subagent instead of one for all of them. Codex after, if someone can tell me what those transcripts look like.
@forgebitz The most fun thing ever is watching "works on my machine" become "worked in my session". Do you ever re-run the same prompt hoping it'll break again so you can watch it?😂
@johnrush AI erased the best part, and I'd narrow it to the writing. Reading and deciding can absorb you the same way, but it's a different job, and nobody spent twenty years falling for that one. Haven't hit sunrise on a code review yet
What moved it for me was banning specific words instead of describing a style. "Write clearly" gets ignored, a list of banned words and no em dashes doesn't. Tying every sentence to an actual change in the diff kills most of the waffle. The bullet list reflex I still haven't beaten.
Yeah, and it goes wrong on the way back up too. What returns is a rollup the subagent wrote about itself, and when I parsed my own session transcripts that's where the numbers drifted most from what actually ran. Does your harness give you the raw subagent output, or only its summary?
@addyosmani Autonomy earned by passing verification loops depends a lot on what the loop reads. An agent's own summary comes out of the same run you're checking, so it can pass for reasons that have nothing to do with the work. How do you decide what a gate has to look at?
9/9
Remember the landing page I keep not taking down?
It's this one: https://t.co/FaYsqxVzVb
Still online, still says "Coming soon".
Something did come. It just wasn't TweeX.
1/9
Back in February I wanted two things: to stop being inconsistent on X, and to make my first euro online.
So I did the most builder thing possible: I built a SaaS about it.
It died four days into beta.
The story is better than the product was. 👇
8/9
Today my X runs on Claude Code: a repo of instructions it reads, drafts it proposes, me approving every single publish.
Zero X API, zero per-seat costs, just the subscription I already had.
What I was trying to sell was a workflow all along.
It all depends on the type of agent. Typically, agent development involves prompts, a knowledge base, tools, and/or MCP servers. If you're developing via code, you obviously have Git for version control; otherwise, for prompts, you have two options: Git (via the filesystem) or Langfuse.
I use Langfuse to version my prompts and manage environments, version tags, or the standard "latest" tag.
Have you ever used it?
@bendee983 That last part is the one I find hard: knowing when a boundary has actually settled. Two runs can land on the same output down completely different paths, so the result alone won't tell you the step is stable. Do you go read the runs, or wait for something to break?
@akshay_pachaar The /refine detail that got me is that it records what triggered the edit. My CLAUDE.md has rules I added after bad runs and I've already lost the reason for half of them, so now I just leave them there. Rollback by ID is the part that would actually get me to clean it up.
Eight candidates for my next open source project. Two adversarial agents on each, one instruction: kill it. Zero survived.
Three died on facts: the thing already existed, in one case at $5 a year. The other five died against a rule I'd inherited from an older plan.
Wrong rule. The goal had moved: I'm not picking a product, I'm picking what to measure and write about. Swapped the criterion and the ranking flipped. The winner was walkaround, sitting on my disk since June.
Asked the model how the auth library handled expired links. Got a confident answer. It was wrong for the version we actually had installed.
So I sent it into node_modules to read the thing instead. Same question, answered off the code that runs, and this time it came back with a file and a line number.
The library had been reporting the real problem all along. Our form wasn't reading that part.
Now when it explains a dependency, I make it open the file first.
They asked me to change one word in a Next.js app, user-facing only.
That same word was doing three jobs in there: a nav item, a role label, a professional qualification. Only one was supposed to change. A find and replace renames all three, and so does an agent if all you give it is "rename X to Y".
I went occurrence by occurrence with a rule for each, and the ones I couldn't call I flagged instead of guessing. 35 UI strings, 30 path references, zero migrations, 846/846 green.
Still don't know how many other words in there are doing the same thing.