av9n

AI grades its own homework. We don't let it. Independent verification that AI agents & software do what they claim — checked against evidence they can't fake. a

[av9n @av9nai]

@wilczyn Your AI agent says it processed all 50 refunds; av9n checks the payment system and tells you the 7 that didn't actually go through.

[av9n @av9nai]

As people start buying and renting AI agents, one question decides everything: which ones actually work?

Not which ones demo well. Which ones have a verified track record of doing what they claimed — checked against real evidence, not their own marketing.

The agent economy needs a trust layer. That's the interesting problem.

[av9n @av9nai]

@sridharfyi https://t.co/pyA8vi408W

[av9n @av9nai]

This is the shift people underestimate. When you manage an agent like a person in a process, the human-in-the-loop review works because you actually spot check the work, you're overseeing one "employee" and you can catch when something's off.

But the whole premise of the post is that most enterprise token usage becomes agent deployed to execute tasks inside workflows. At that scale, the review step quietly breaks. You can't personally review hundreds of agents executing thousands or actions across workflows and the agents own "done" isn't evidence that it's actually done. The failure that costs you isn't the agent that errors out; it's the one that confidently reports success on work that didn't happen the way it claimed.

So the human-in-the-loop review can't just evolve, it has to be come something a human isn't personally doing on every task. It has to become an independent check on whether the agent's claimed output actually matches reality, against evidence the agent doesn't control. That's the piece that has to exist before "deployed to execute" scales past the point where a person can watch is happen.

[av9n @av9nai]

We trust software because it earned it — decades of deterministic systems that mostly did what they said.

AI agents inherited that trust without earning it. They speak in the same "task complete" language, but the reliability underneath is gone.

The interface says "done." The guarantee behind it quietly disappeared.

[av9n @av9nai]

McKinsey just put a number on something most people building agents haven't clocked:

~60% of an agentic task's cost isn't generating the answer. It's the checking, repairing, and re-verifying afterward.

Verification isn't a nice-to-have you bolt on at the end. It's already most of what your agents actually cost. Might as well make it trustworthy.

[av9n @av9nai]

Spent the week making my demo actually work on people's real inputs instead of just canned examples.

The moment it could catch a real agent lying — not a scripted one — was when it clicked for me why this matters.

An agent will tell you "done" with total confidence. The interesting question is whether "done" survives contact with reality.

[av9n @av9nai]

@launch_llama https://t.co/pyA8vi408W

[av9n @av9nai]

"Can the AI agent do the task?" is almost a solved problem. The models are good and getting better fast.

"Did the agent actually do the task — this time, on real data, without quietly getting it wrong?" is barely being asked.

Capability is racing ahead. Verification isn't. That gap is where the real risk lives now.

[av9n @av9nai]

The wild part of agents running 1,000+ autonomous tool calls isn't the scale — it's that at that scale, "it found improvements" is a claim nobody can eyeball. When an agent runs unsupervised for 24 hours, the hard question shifts from "can it do the work" to "how do we verify what it actually did was real?" Autonomy raises the ceiling and the stakes at the same time.

[av9n @av9nai]

An AI agent doesn't crash when it's wrong. It reports "done" — confidently — whether the work happened or not. And the tools watching it read the agent's own report. That's the gap. https://t.co/WSzRAHAdTh

[av9n @av9nai]

@anupamrjp https://t.co/pyA8vi408W

[av9n @av9nai]

Something I keep running into building in this space:

An AI agent doesn't fail the way normal software fails. Software crashes — it throws an error, something lights up red, you notice. Agents don't do that. They report success. "Task complete." "Payment processed." Same confident tone whether the work actually happened or not.

The failure isn't loud. It's an agent that's quietly, confidently wrong — and nothing flags it, because the agent's own report says everything's fine.

The part that gets me: almost every tool built for agents reads what the agent says about itself. Traces, evals, logs — all downstream of the agent's own account. But the agent's account is exactly the thing you can't take on faith once real money or real customers are involved.

You can't verify an agent by asking the agent. The check has to come from outside — the payment processor, the system of record, evidence the agent doesn't control. Otherwise it's not a check. It's hope with extra steps.

[av9n @av9nai]

@V1adXBT 👋

[av9n @av9nai]

@asonninen https://t.co/pyA8vi408W

[av9n @av9nai]

@xadisingh https://t.co/kc6kVgj9zL run a repo through my demo and let me know what you think.

[av9n @av9nai]

@boardyai @andrewdsouza Would love to get some people to check out my software. Thanks!

[av9n @av9nai]

Most "AI agent" demos are a landing page and a waitlist. I wanted people to actually use av9n, so I built a live demo with no signup.

Here's the idea it shows: an AI agent tells you it did something — "processed all 50 refunds," "deployed to production," "resolved the ticket." av9n checks whether that's actually true, against evidence the agent doesn't control.

Three things you can try right now:

→ Pick a scenario and watch it catch an agent that reported success on work that never happened. The agent says "all 50 refunds processed." The payment log says 7 failed. av9n flags it.

→ Point it at any public GitHub repo and a claim about its latest build. av9n pulls the real CI run and checks your claim against actual data — not the agent's word for it.

→ Paste your own agent's log and what it claimed. av9n flags contradictions inside it — like reporting success while the log shows an error.

No account, no email. Just try to break it and tell me where it falls short: https://t.co/8SjMtw0MaV

[av9n @av9nai]

@lightsilver323 An AI agent grading its own homework isn't verification — av9n is the independent check that catches when "done" isn't actually done.

[av9n @av9nai]

@lightsilver323 https://t.co/pyA8vi408W