Federico

Software & AI Engineer. I ship real systems with AI agents and post what breaks on the way.
Employee → builder, in public.

npx walkaround

[Federico @DevCalledFede]

Ran my own tool on my own machine last night and found 463 errors I'd never looked at.

Built walkaround back in June. It reads Claude Code's local transcripts and reports what the agent actually did to your repo. 40 sessions on this laptop. 113 errors in the headline numbers, the ones I read every time. Another 463 sitting inside the subagent rollup.

I'm the one who put that rollup in its own section, so sessions with and without subagents would stay comparable. Then I stopped opening it.

Fixing that next: one line per subagent instead of one for all of them. Codex after, if someone can tell me what those transcripts look like.

[Federico @DevCalledFede]

It all depends on the type of agent. Typically, agent development involves prompts, a knowledge base, tools, and/or MCP servers. If you're developing via code, you obviously have Git for version control; otherwise, for prompts, you have two options: Git (via the filesystem) or Langfuse.
I use Langfuse to version my prompts and manage environments, version tags, or the standard "latest" tag.

Have you ever used it?

[Federico @DevCalledFede]

@bendee983 That last part is the one I find hard: knowing when a boundary has actually settled. Two runs can land on the same output down completely different paths, so the result alone won't tell you the step is stable. Do you go read the runs, or wait for something to break?

[Federico @DevCalledFede]

@akshay_pachaar The /refine detail that got me is that it records what triggered the edit. My CLAUDE.md has rules I added after bad runs and I've already lost the reason for half of them, so now I just leave them there. Rollback by ID is the part that would actually get me to clean it up.

[Federico @DevCalledFede]

Eight candidates for my next open source project. Two adversarial agents on each, one instruction: kill it. Zero survived.

Three died on facts: the thing already existed, in one case at $5 a year. The other five died against a rule I'd inherited from an older plan.

Wrong rule. The goal had moved: I'm not picking a product, I'm picking what to measure and write about. Swapped the criterion and the ranking flipped. The winner was walkaround, sitting on my disk since June.

[Federico @DevCalledFede]

Asked the model how the auth library handled expired links. Got a confident answer. It was wrong for the version we actually had installed.

So I sent it into node_modules to read the thing instead. Same question, answered off the code that runs, and this time it came back with a file and a line number.

The library had been reporting the real problem all along. Our form wasn't reading that part.

Now when it explains a dependency, I make it open the file first.

[Federico @DevCalledFede]

They asked me to change one word in a Next.js app, user-facing only.

That same word was doing three jobs in there: a nav item, a role label, a professional qualification. Only one was supposed to change. A find and replace renames all three, and so does an agent if all you give it is "rename X to Y".

I went occurrence by occurrence with a rule for each, and the ones I couldn't call I flagged instead of guessing. 35 UI strings, 30 path references, zero migrations, 846/846 green.

Still don't know how many other words in there are doing the same thing.

[Federico @DevCalledFede]

@yacineMTB Watching the reasoning is half of it. The other half is the tool calls, which you get from every model, and when I lined the two up across my own sessions they didn't always agree. The summary is confident in a way the trace doesn't back up.

[Federico @DevCalledFede]

@SherryYanJiang For me it splits on verification, not on setup. If the diff plus a green test run tells me it worked, cloud is fine. If I need to see what it touched along the way to trust it, it stays local. Nothing embarrassing about that half, by the way.

[Federico @DevCalledFede]

@quxiaoyin Agreed, and it's the part nobody really measures. What I end up looking at is how much work an agent repeats inside one session. Swapping the model changes that number less than swapping the harness does. Curious what you judge one on at Tycoon.

[Federico @DevCalledFede]

https://t.co/dotLznSAcM<your-package>

The number on the package page is a weekly total. It hides the shape.

[Federico @DevCalledFede]

npm says walkaround got 124 downloads last week. 122 of them landed on Monday, the day I published 0.2.2. The other six days added up to two.

Yesterday I posted about the tool for the first time. npm logged zero.

So the spike came a day before anyone heard about it. That's registry mirrors and security scanners, and they grab every new version within hours of a publish.

If one day carries your whole week and it's the day you shipped, those aren't people.

[Federico @DevCalledFede]

@sergeykarayev The boundary question is the right one, and it costs something on the work side too. A cloud job of mine kept dying because the authenticated browser session it needed only lived on my laptop. How do you get a session into a sandbox without handing over the credential?

[Federico @DevCalledFede]

@rtfeldman Building the feature is fine, finding the input that breaks it is suspicious. Same code, same session, and the read flips the moment the task becomes "make this fail". I'd try calling it a regression test and see if that clears customs.

[Federico @DevCalledFede]

@kunchenguid The escalate-vs-go-rogue trait is the one I'd have picked too, and it's also the hardest to see: when a crewmate goes off, the second mate's summary still reads fine. The diff is where it shows up. How do you catch it, summary or transcripts?

[Federico @DevCalledFede]

@bthdonohue The "seven times" is the agent counting itself, which is what gets me. I ran 40 of my own sessions back through the raw transcripts and the headline numbers came out wrong 113 times. Did yours really go seven, or did it round up for the story?

[Federico @DevCalledFede]

@Saanvi_dhillon Yeah, most weeks I'd skip the new acronym too. This one I ran both ways though, same content as JSON and as TOON. On long structured input the TOON version cost less and the model kept the fields straight. On short stuff I couldn't tell the difference

[Federico @DevCalledFede]

@desphixs Yeah, way more. My last deep research fan-out ran 102 agents and 2.48M tokens in 21 minutes and came back with 5 lines. Absurd on paper, and I'd pay it again, one session was never getting there

[Federico @DevCalledFede]

@timClicks The incentive's real, it just doesn't predict where the wordiness shows up. It spikes where the model's least sure, and it mostly dies the moment you ban hedges and preambles. Weird thing for billing to obey a word list.

[Federico @DevCalledFede]

@shannholmberg Same harness across models is the part I'd sit with. Describing the voice never held for me. What held was banning: word list, no em-dash, hard cap on hedges. Overexplaining and made-up terms were the first two through the gap. Does yours forbid anything?