An average is not a comparison.
Fork a run that happened with one declared change. Steps before it replay from their recorded results; execution resumes at the change. Same input, one difference, both scored.
https://t.co/ESQ2KsgVwN
One run is one trace id.
Every step span joins it - across retries, across the runners it touches. Exported by the OpenTelemetry provider you already register, into the backend you already run.
No second observability system.
https://t.co/NjhjFnbJMc
What did the agent actually cost?
Spend by model, by workflow, by hour. Tokens always, cost once you price it. Latency and cache-hit rate beside them.
Metered by the runtime that made the calls, not estimated after.
https://t.co/NjhjFnbJMc
Your provider quota is a rate. Your agents burst.
A token throttle holds the line: actual tokens debited as steps commit, and over the line, new runs start later - spread out, queued, holding no worker.
Nothing fails. The rate bends.
https://t.co/NjhjFnbJMc
Change the model. Replay the same run.
Fork a scored run with one declared change: steps before it replay from recorded results, execution resumes at the change. Two runs, one difference, both scored side by side.
Fork the run, not the average.
https://t.co/ESQ2Kshtml
The model returned a 429. The run kept going.
A declared fallback chain advances the call on a rate limit, 5xx, or timeout; the first model to answer wins. One result recorded, with which model served and why.
https://t.co/NjhjFnchBK
Where did this run come from?
Every ingested event is kept with what it triggered: app, source, outcome, down to "1 run". An event that matched nothing is recorded too.
Each triggered run links back to its exact event.
https://t.co/NjhjFnchBK
Same model, same prompt, same params. Why pay for it twice?
An identical call short-circuits to the cached answer: the provider is never called, zero tokens, nothing counted against the cap.
Deterministic calls only. Off by default.
https://t.co/NjhjFnbJMc
@alexxubyte Most of the failures weβve seen are in the constraints and the verification steps.
The agent tries to do something risky, nothing really stops it, and by the time you check the result itβs already done.
No human in this video, on purpose.
The run hits its cap and fails before the call that would cross it. Checkpoint kept, spend recorded up to the halt.
https://t.co/WKhj7jkNYM
Two stops, no human in either:
a cap fails one run before the call that would cross it.
a budget pauses the whole window, then resumes on its own.
https://t.co/WKhj7jkNYM
Do not record where the run was. Record what has finished.
A step has a stable id. The first pass executes and writes the result down. Every pass after - crash, deploy, replay - returns the recorded result.
https://t.co/ZCuxeSGOVI
Inside one agent turn: the model decides, the tool runs, and both land in the journal with a checkpoint.
That record is what makes replay, resume, and caps possible.
https://t.co/NjhjFnchBK
One run, thirteen steps: load the ticket, classify it, triage with one durable re-ask, embed the knowledge base in batches, an agent loop with its tool calls, send the reply.
Every step in the chain, in the order it ran.
https://t.co/NjhjFnchBK
Your agent wants to issue a refund.
It parks. No worker held, no thread waiting - the run is checkpointed at its gate.
A person reads the args and approves. The run resumes from that exact step.
That is what ask-first means.
https://t.co/WKhj7jllOk
An agent is opaque while it runs. You find out it went wrong once it already has.
So we made the run the thing you watch: every step, the tool call, the arguments it chose, the moment it stopped.
https://t.co/S0nv5m6agy
Step 4 issued a refund. Step 7 crashed.
You resume from a perfectly consistent checkpoint at step 4.
Does the refund happen again?
Only if step 4's result was recorded, not just the fact that step 4 was reached.
https://t.co/ZCuxeSGOVI
What if the agent does something expensive?
Most stacks answer that with an alert. But an alert is a description of something that already happened. The spend has been spent.
Enforcement is a different thing: the run stops, and waits for a human.
https://t.co/WKhj7jllOk