I'm building INFRO (https://t.co/sD6bUBhNcR), the control plane for production AI.
You keep your own provider accounts, contracts and keys. INFRO sits in front of them: routing, failover, spend caps checked before the call, and one view of what every team and customer costs.
0% markup on model spend. Your providers bill you for AI; INFRO bills you for the software.
Here I'll post what running AI in production really costs.
Every call through the gateway, with its trace: model, project, status, latency, tokens in and out, estimated provider cost.
Search by request id, trace, user or tag. (demo workspace)
INFRO Studio: images, video and audio on your own provider accounts.
You see the estimate before you generate, which provider runs first and what it fails over to. Your provider bills you; INFRO adds nothing to it.
Flows chain the steps once and run them on a whole list. (demo workspace)
@Sonofpeace0001@DanKulkov checkpoint file per step is the right call. once that exists you can also resume on a different model when the limit hits instead of waiting out the reset
Every route in INFRO is a primary and its fallbacks, across the providers you already pay:
OpenAI โ Vertex
Anthropic โ Bedrock
OpenAI โ Azure
The console counts what each fallback recovered. (demo workspace)
@GergelyOrosz The second group often isn't short on autonomy, they're short on permission: no budget for tools, no rule on which models are allowed, so trying things feels risky. Are you seeing companies close the gap just by writing those rules down?
@SebastianRoehl The ChatGPT line is the story: 52% click-through vs 26% overall. People who arrive from a ChatGPT answer already have the recommendation, so the page only has to confirm it. Can you see which questions send them, or is that still a black box?
@mattpocockuk The one force that doesn't go to infinity for free is the bill: every failed lint, type or test check is another loop with the full context re-sent. Have you hit a point where stricter rules cost more retries than they save?
@hwchase17 On 2/: when I review a slow or expensive agent run, the answer is usually one step that retried and re-sent the whole context. Does the Trajectory view show cost and retries per step, or is that a separate screen?
@sermakarevich The "decide" step is the line people miss on the bill: it's a second model call on every turn, including the turns where nothing gets kept. Did any of the 30 let you run that extraction on a smaller model than the agent itself?
What a team running AI in production sees in INFRO (demo workspace, sample data):
- estimated provider spend against budget, per project
- unusual spend flagged the same day, with what drove it
- limits that block requests before the provider call, not after the bill
Your providers still bill you directly. 0% markup on model spend.
@bukanajay Hi from the infra side. I'm building the layer between apps and the AI providers they already pay: routing, failover, spend caps. What breaks first when your agents meet real users?
@LangChain Does the harness pick a model per step, or one model end to end? Video, graphics and decks have very different cost per output, so I'm curious how you route them.
@heyemilyx I'd draw the line at reversibility. Drafting is free, a sent email isn't, so sends wait for a human until the agent has a track record on that exact kind of email. How many a day are we talking?
@0xbobaaa Per-task cost is the right comparison, not per-token. Did the benchmark count reruns? A model 3 points behind can win or lose depending on how often a miss needs a second try.
@DamiDefi The retry line is the one I'd watch. A retry after a timeout re-sends the full context, so one flaky tool can double a run's cost. Does the trace mark retried calls separately?