What stuck with me: nothing shipped on the model's word. People confirmed the finding and tested the fix first.
AI agents in production need the same. When one says it issued a refund, check the app's records.
That's what I'm building ScopeHaven to do. How do you check yours?
Over the past two months, we have been leading a coalition to mitigate one of the most severe bugs in open source software found in recent memory, php2shell, found as part of our Patch the Planet initiative. https://t.co/rBsz7lrlwo
Your AI agent said it refunded $20. It never happened.
ScopeHaven catches that by checking your agent against your own records.
Our test app: 26 of 44 checks failed. After the fix: 44/44, 0 new failures. Runs locally.
(Simulated UI, planted bugs, synthetic data)
Your AI agent refunds a customer $30.
It sends $3,000.
Then it reports: "Done! $30 refunded." 💀
Would your team catch that before your customer did?
I'm 19 and building ScopeHaven to catch exactly this.