3/ A couple of rules we’ve ended up with:
If you fix a bug, revert the fix and make sure the new test actually fails.
And if an agent writes an identifier, enum, event name, whatever, check it against the source of truth. Don’t trust it because it looks right. Mechanical gates!
1/ We use coding agents a lot now, and a weirdly consistent failure mode keeps showing up.
The bad output usually doesn’t look bad.
One agent invented three identifiers that weren’t in our schema. All three looked completely real. The code ran. It just returned fewer results.
2/ Another wrote a test that stayed green for two months because the fixture could never reproduce the bug.
Nothing crashed. Nothing looked obviously suspicious. That’s what makes these annoying.
2/ We replaced the prohibition with an example of the kind of answer we actually wanted, and the behavior stopped.
A useful lesson when building MCP is that every instruction you give a model is also language it may imitate.
1/ We gave agents via our MCP a rule meant to limit what it could confidently claim.
Instead, it started repeating the rule back in its answers.
It followed the instruction perfectly.
We had written a constraint in language that was too easy to treat as user-facing copy.
2/ We replaced the prohibition with an example of the kind of answer we actually wanted, and the behavior stopped.
A useful lesson when building MCP is that every instruction you give a model is also language it may imitate.
@Muse@Shopify@PayPal@Expedia@Instacart@muse filed a bug.
custom OAuth connector setup demands a hand entered Client ID even when the provider supports automatic client registration.
@finkd https://t.co/HESND4Ap3A is live on @Muse. Ask what to study for the SAT and it answers from the questions you've actually practiced, not a generic plan.
@Tesla Found a work around, submit a random maintenance request, get to scheduling, confirm the appointment; and then add additional issues to it, now you can call back to reschedule the date!!
Never this frustrated scheduling a service taj with @Tesla! Car stops functioning after an update and now I am in an infinite loop of diagnosing in app with only option eventually is to call service center who can't book anything with in a week as that's all system allows them l.
This is so backwards and creates mistrust about the auto diagnosis. CS literal suggestion now is just drive your car with all disabled functionality to the nearest service center!! No proximity warning ability or anything, like a 30 year old car!! Make it better please @elonmusk
The orchestration logic in @garrytan’s gstack is underrated.
The built-in compacting phase before every plan execution is the real win, it forces a project state sync at the start of every session, effectively killing context rot in long horizon agentic workflows.
@garrytan One issue I ran into was my /plan-ceo-review session got dropped while working through few questions.(Claude code has severe issues with Snapdragon+Windows 😔).
But when I restarted the session, the CEO review had no memory of it. Luckily I had the earlier chat logs.
Just found out "/model opusplan" exists in Claude Code.
That is a pretty smart setup, Opus for planning, Sonnet for implementation.
A lot of practical AI usage is basically this. Pay for depth at the decision point,
not on every line of execution.
3/ Kept the important architecture context. Moved the stale stuff out. Still had Archive.md when needed.
At some point context files need maintenance too.
1/ One thing I ran into on a bigger project is Claude.md started getting too big.
On restart, Claude actually flagged it and asked me to cut it down. I did not want to manually clean a long markdown file,
So I told it:
2/ “Review Claude.md. Move completed tasks, rejected ideas, and implemented technical tasks to Archive.md. Keep Claude.md focused on active development and core constraints.”
It handled that better than I expected.
1/ Anthropic’s managed agents post is interesting for one reason, it assumes the environment is chaotic.
That’s what most real enterprise settings look like too. Things fail half way. Context gets too long. Credentials cannot be everywhere.
https://t.co/lqKLk6C0ls