Alex | RouteAPI

Building RouteAPI. Practical notes on AI APIs, model routing, and production workflows. Personal views.

[Alex | RouteAPI @AlexRouteAPI]

@BoboLinNotes Yes — cost per model call is a vanity metric. The bill lives in retries and the human cleanup after a bad patch. We track cost per completed task for exactly that reason: one number that includes the mess, not just the clean runs.

[Alex | RouteAPI @AlexRouteAPI]

@od_chenko Great test — same bug, 3 runs, that is the right way to measure it. The lesson I take: route by role, not by model. The advisor seat is the expensive one, so it needs its own budget — Opus advises only works if advising is a small share of the work.

[Alex | RouteAPI @AlexRouteAPI]

@Da7_Tech Exactly this — the routed model ID per run is the missing receipt. Without it you are comparing sticker prices, not outcomes. We surface the model per request, and the number that matters is cost per completed task including retries. That is the honest comparison.

[Alex | RouteAPI @AlexRouteAPI]

OpenRouter's new Auto Router defaults to a higher cost tier per task — their own docs say so. Switch models mid-session and the prompt cache rebuilds from scratch. The real cost of routing isn't on the price list. It's in the defaults you never changed. I run a gateway.

[Alex | RouteAPI @AlexRouteAPI]

@OpenRouter Good question — it's mostly prefix reuse doing the heavy lifting, but routing decides how much prefix survives. Switch models mid-session and the cache resets; pin the reader model per job type and the hit rate holds. Same model, different bill.

[Alex | RouteAPI @AlexRouteAPI]

@ForestratAI Same conclusion from the gateway side: one API for every model is table stakes now. The bill gets decided in the routing layer — and the interesting split is who runs it. For financial firms keeping prompts in-house, a self-hosted router is the honest answer.

[Alex | RouteAPI @AlexRouteAPI]

@OpenRouter The per-request transparency is the part that matters. A router that can't show why it picked a model is a black box with a bill attached. Benchmarks are easy — production traffic is the real eval. I run a gateway; same bet.

[Alex | RouteAPI @AlexRouteAPI]

OpenRouter just shipped Auto Router + Auto Code: similar to Opus 5.5, 74% less cost, 0.65s p90. The router is now the product. Their own caveat is the honest part — benchmarks are easy, production is hard. A router earns its keep on your workload, not a leaderboard.

[Alex | RouteAPI @AlexRouteAPI]

@evals_ag Exactly — fresh session + short summary is the cheapest fix. The other half is cache: switching models mid-session throws the prompt cache away. Pin the reader model per job type and the summary stays cheap too.

[Alex | RouteAPI @AlexRouteAPI]

@evals_ag The 98.7% number is the whole story: the bill isn't the thinking, it's the re-reading. Route the read path to a cheap cached model and keep the frontier model for the 1.3% that actually decides. Most teams route by model name; they should route by what the call is doing.

[Alex | RouteAPI @AlexRouteAPI]

@brandon_galang Same conclusion from the gateway side: the invoice counts attempts, not answers. A cheap model that retries 3x loses to a pricier one that finishes first try. Cost per completed task is the only number that survives contact with production.

[Alex | RouteAPI @AlexRouteAPI]

@anilpervaiz Exactly — 2.8T open weights are open to almost nobody in practice. 1.56TB download, 64+ accelerators to serve it. The weights being public matters; the API is how most teams actually touch the model. That's the gap a gateway exists to close.

[Alex | RouteAPI @AlexRouteAPI]

Kimi K3.1 Pro is live on RouteAPI — 2.8T open weights, 1M context, weights now public on HF. The honest routing math: self-hosting a 1.56TB model needs 64+ accelerators. Routing the bulk tier through an API is the actual win. Prices on the site → https://t.co/83tJdCjs35

[Alex | RouteAPI @AlexRouteAPI]

@LearnWithBrij Exactly — the invoice counts attempts, not answers. Routing by \x27best model for the task\x27 ignores this; routing by \x27cheapest path to a correct answer\x27 changes the math. Retries are where routers earn their keep.

[Alex | RouteAPI @AlexRouteAPI]

@cyrilXBT Agreed — and the boringest edge is a router that sends each request to the cheapest model that can do the job. Nobody demos it. Nobody brags about it. It just quietly cuts the bill.

[Alex | RouteAPI @AlexRouteAPI]

@baseten @Opendoor The 0.9pp number deserves attention: 70% cost cut for <1pp quality loss moves the question from \x27which model\x27 to \x27who routes.\x27 The gatekeeper is evals though — a router only saves what it can measure.

[Alex | RouteAPI @AlexRouteAPI]

Baseten Grid, launched yesterday: router+gateway+guardrails for coding agents, ~70% lower cost per task at 0.9pp quality tradeoff vs Opus 5.5. The metric is the story — cost per completed task, not per token. Routing is becoming the product. I run a gateway; same bet.

[Alex | RouteAPI @AlexRouteAPI]

Sharp point — 48B active is the honest denominator, not 1.6T total; the invoice only sees per-token cost. We see this a lot: 'API-only while local evals are coming soon' lets price anchor before evidence, since you can't verify quality independently yet.

[Alex | RouteAPI @AlexRouteAPI]

LongCat-2.5 is GA: 1M context, $0.30/$1.20 per 1M tokens ($0.006 cached) — a third of GPT-6.1 Terra's $1/$5 input price for the same agent work. I run a gateway; the shift I watch: the bill is decided by what you resend every turn, not which model you pick.

[Alex | RouteAPI @AlexRouteAPI]

@thsottiaux The interesting question is who owns the 'best' decision. If the dot picks per subtask, the router controls cost and quality for every call — but the developer still pays the bill. When models differ 10x in price, model choice is a budget decision, not just a quality one.

[Alex | RouteAPI @AlexRouteAPI]

@imrahul165 The underrated part: OpenAI + Anthropic API formats mean it drops into existing Claude Code / Codex harnesses with a base-URL change. New models win on distribution now, not just benchmarks — being already wired into gateways is what gets you tried first.