TokenRouter

One OpenAI-compatible gateway in front of 13 providers. Budgets with hard caps, rate limits, and analytics. Flat subscription, zero token markup.

[TokenRouter @tokenrouter]

@pevidex @josephfcox 3 prompts and done is wild. was that opus with a fat context, or something else chewing the window?

[TokenRouter @tokenrouter]

@Merstradamous the %/hr jump after a global reset is the part that drives people nuts. are you measuring against the same model and effort, or did the harness change too?

[TokenRouter @tokenrouter]

@code_kartik yeah the plan bar is a terrible predictor of real burn. we put hard daily caps on each engineer key at TokenRouter so it 429s before the call. were those 5 long-context prompts?

[TokenRouter @tokenrouter]

what a coding-agent week looks like when every request goes through one gateway

spend, requests, blocked, failed. per team. no markup on your own provider keys

[TokenRouter @tokenrouter]

@Alvin_Abiero same class of bug. the agent happily reports success from a stale tool result. we sample those trails with a tiny judge rubric on "did the side effect actually happen" instead of trusting the summary

[TokenRouter @tokenrouter]

@trevin @bokuwalily yeah the silent cost is wall clock. we started logging hook duration next to token spend and found one stop hook eating minutes for almost nothing. do you kill hooks that never pay for themselves?

[TokenRouter @tokenrouter]

@tristanbob api overflow with a hard daily cap. we put ours behind TokenRouter so leftover spend can't runaway overnight. you sticking with one provider or mixing?

[TokenRouter @tokenrouter]

LLM-as-a-judge on live coding-agent traffic, not a batch eval later

Hallucination check sampled 10% of org traffic. today: 4 judged, 2 flagged — including a medium hallucination on a made-up Python 3.14 flag. pass/fail + reason on each call

same gateway that routes and budgets the request

[TokenRouter @tokenrouter]

Route Claude Code and Codex through one gateway. One key, one daily hard cap, one log per engineer, whichever agent they use.

https://t.co/T52mLLg0bn

[TokenRouter @tokenrouter]

spend caps in tokenrouter stack at four levels: org, team, engineer, key

so the platform team can have a monthly budget, each engineer a daily cap inside it, and a runaway codex session still stops at its own key

all four get checked before the request goes to the provider, not after the bill shows up

[TokenRouter @tokenrouter]

@ashtonamory agreed. pausing to ask beats guessing and redoing the work. we also like it for the same reason we like a cap: find out at a checkpoint, not after the damage.

[TokenRouter @tokenrouter]

@quinneunderwood it is fun once the rubric exists. we sample live traffic async and run it through a judge with a plain-English rubric, so failures show up as readable cases instead of a benchmark score.

[TokenRouter @tokenrouter]

@Jason a usage-based chat bot without a ceiling will always surprise you. we put a hard per-agent daily cap on the key, checked before the call goes to the provider, so a chatty slackbot burns to a wall instead of to a bill.

[TokenRouter @tokenrouter]

@LeeLeepenkman @bcherny usage walls mid-compose are common. we draft long prompts elsewhere and copy them in so a limit hit never eats the text. also worth a hard daily cap before the call so you find out at a checkpoint, not mid-sentence.

[TokenRouter @tokenrouter]

@cudanexus @Anthropic @claudeai we track cost per finished task instead of raw tokens, plus a hard daily cap on our own key checked before the call goes out. a burn spike can't run overnight that way. that is the setup we built tokenrouter around.

[TokenRouter @tokenrouter]

@ListenJet cost per finished task with the harness held constant beats another round of model debates. hold the workflow still and the burn numbers usually pick for you.

[TokenRouter @tokenrouter]

@stretchcloud the clean 200 hiding a dead end is the nastiest version of this. sampling those sessions through a judge with a plain english rubric catches it without touching the request path.

[TokenRouter @tokenrouter]

@xiaoshan0413 the rubric has to be written for the whole task, not each step in isolation. sampling real production transcripts into the judge is usually a faster start than generating 500 synthetic cases.

[TokenRouter @tokenrouter]

@manvankaliber two subs burning through the same window is rough. a per-key rpm cap spreads the load so the second one does not die in 40 minutes.

[TokenRouter @tokenrouter]

@lazyynocturnal the reset only helps if you know what you have left before you start. a cap you set per key makes the number boring instead of a surprise.