Your AI agent is one tool call away from an unauthorized refund or doing something totally stupid.
Failproof blocks the call, shows the agent the right policy, and finds similar failures hiding in past traces.
20 people opened our dashboard at once. Every page returned a 504. A few weeks later, the same 3 CPUs serve 200 concurrent users with zero errors.
@SiddarthAhhaaa walked through exactly how in our latest engineering session. The fixes, with the numbers 🧵
20 people opened our dashboard at once. Every page returned a 504. A few weeks later, the same 3 CPUs serve 200 concurrent users with zero errors.
@SiddarthAhhaaa walked through exactly how in our latest engineering session. The fixes, with the numbers 🧵
got my own ticket for hacktober hack 🎟️
this sunday, 100+ builders, one ai agent. every team gets the same one and has under two hours to make it something you'd actually ship.
sun 25 oct · 4:30 pm · quad ai, hsr bengaluru
@failproofai@digitalocean@MLHacks
Hacktober Hack 🔥
700 signups. 100 spots.
Everyone gets the same flaky AI agent. 2 hours to make it shippable. Then a hidden eval suite runs it 3 times per task. Pass all 3 or it doesn’t count.
HSR Layout, Bengaluru.
Your AI model can't actually do anything. The harness does.
The model writes text. The harness runs it: on your machine, with your keys, against your real database.
The one moment you can still stop it: after the model asks, before the harness acts. Steer there.
AI moves so fast that we have newspapers covering it now.
Got my copy of The Token Times: a pretty cool way to document everything happening at the frontier.
Shoutout to @failproofai and @niveditjain for this one! 🗞️
"Just put it in the prompt" is a polite request.
A prompt is a polite request competing with thousands of other words.
Logs? After the damage.
A human? Not at 3am.
Steering is a decision between the model and the tool, before anything runs. Every single time.
handling 10 million agent traces a day!
Anyone can build an agent tracing demo at 1,000 traces a day.
Plenty of tools look great until the traffic shows up. We’re at 10M agent traces a day, and getting there was not child’s play.
@SiddarthAhhaaa is breaking down how we store and query at that scale, including what broke along the way.
Oct 20, 6pm PT, Gmeet!
Agent's done != done
Finishing is an action, so a policy can check it. Before the loop ends, the agent must ask GitHub. APPROVED: done. Anything else: blocked.
And if GitHub can't be reached, it blocks too. Steer on what's true, not what agents remember.
Our AI agent said the PR was approved. It wasn't.
Our Hermes agent had one rule: review the PR and don't leave the loop until it's approved. The GitHub token had expired and the request timed out, but the agent saved "approved" to its memory and moved on. The loop believed it, because the agent was the only thing it was listening to.
Nobody did anything wrong. The outcome was still wrong. Part 2 shows the fix.
From our engineering session "How We Steer Hermes" with @Chhhee10
500+ registrations.
MLH said 50 check-ins. 😭✌️
We said… nah. 💀
Somehow got 150+ builders into the venue, shipped crazy shit, and ran 3 sessions on Hacktoberfest, Open Source & Devcon.
Hosted this absolute chaos with FAIR and had the time of my life.
Crazy successful.