17 agents run our company. 11 of 50 incidents in a week came from our own manual, not the model. The cockpit is open source, MIT. https://t.co/mKk8VOfLNh
@GavinCampbellAI Same rule at company level for us: each agent owns its folder and can read the others', never write them. Every mandate says what the agent must not touch.
Build in public, one week of data: we run the company with 17 AI agents. Our incident log had 50 entries, and 11 were agent mistakes that came from our own manual, not the model: a rule ignored, outdated data, information missed, a file in the wrong place. When something breaks we change the rule, not just the answer.
17 AI agents run our company. In one week of our incident log, 11 of 50 incidents came from our own manual, not the model. We open-sourced the cockpit (MIT) and wrote up what we learned: https://t.co/1mxMxEFedZ
17 agents run our company. 11 of 50 incidents in a week came from our own manual, not the model. The cockpit is open source, MIT. https://t.co/mKk8VOfLNh
@Stellar_Civi Yes. The manual is files in a git repo, so every rule has a date and an author, and we can diff what changed before an incident. Each rule also carries the reason it exists.
17 agents run our company. 11 of 50 incidents in a week came from our own manual, not the model. The cockpit is open source, MIT. https://t.co/mKk8VOfLNh
@mathiasvrb@grok Yes. The instructions are files in a git repo, so every change has a date and an author and we can diff what changed before a failure. Each rule also carries the reason it exists, next to the rule.
A week of our incident log, running a company with AI agents: 50 entries.
11 were agent mistakes that came from our manual, not the model: a written rule ignored, outdated data used, information missed, a file in the wrong place.
The part of the failure we can fix ourselves is the manual.
A question for anyone paying for five or more SaaS tools:
If someone built your own version of them, hosted it, maintained it, and charged about what you pay now, would you switch?
What would stop you?
@alexander_jais We keep the same thing: a table of what we closed on purpose and when, so nobody, human or agent, rebuilds it by mistake. The "ruled out" part is the one that earns its keep.
@datachad Agreed. The hard part is defining "normally returns some": that baseline has to live somewhere other than inside the agent, or it learns to expect zero.
@chriswiser The support inbox. In the case we automated, it was one person, 58 hours a month: big enough to measure, small enough to be safe. Measuring the before is the step most people skip.