5k fake signups. $40k AI bill by morning.
Every AI founder I talk to has a version of this fear. So I built Noburn.
Rate limits aren't budgets.
A rate limit says "slow down."
It doesn't know a user already burned $300 today.
Provider alerts are worse: they email you after the money's gone.
Noburn is one check() in your own code, before every LLM call. Under budget, it goes through. Over budget, it never leaves. No proxy, so your prompts never route through anyone else's servers.
I tested it in production with a $0.05 budget. It held.
The surprise was loops.
Agents rarely overspend in one big call. They retry the same failing tool 40 times.
So we added loop detection: in our tests, 19 of 20 stuck runs changed course, and task accuracy went from 70% to 85%.
Now I want proof from real products, not my tests.
I'm looking for 15-20 teams shipping AI features to work with directly. We set it up together, test it end to end on your app, and find out honestly whether it solves your problem or not.
Free until mid-December, open source (MIT). If you're interested, reply "noburn" with yoir usecase and I'll reach out.
Strong point. Oracle best-of-N is an upper bound, not a real baseline.
Even with the same fixed checker, the key question is whether coordination improves the candidates the selector sees, or if most of the gain is just better selection that could be added to independent attempts.
multi-agent orchestration needs a better baseline than “one agent with a worse prompt.”
if five agents beat one, that doesn't tell us why.
was it the task split? independent attempts? a reviewer catching mistakes? or just more tokens and tool calls?
i'd want to see:
- one agent with the same total token budget
- multiple independent attempts without coordination
- the full orchestrated setup
then compare success rate, latency, and cost per successful task. count failed runs and coordination overhead too.
orchestration might genuinely help. but “more agents scored higher” isn't enough to show that the coordination was worth paying for.
what's the strongest evaluation you've seen that separates those effects?
@AnnaBlazickova No, not like that.. Actually we have customers ho use legacy software and dont want to share their data with frontier models. For our internal work we use grok bot. But in our market facing product we need to support n8n.