LiteLLM (YC W23)

The industry standard AI Gateway for models, mcps and agents

[LiteLLM (YC W23) @LiteLLM]

Gemini 4 Hydrogen from @GoogleDeepMind: $1.50 In/ $7.50 Out per 1M tokens, and it surpasses Argon on the AA Intelligence Index.

1M context, 2.5x faster than 3.8 Flash.

Supported on LiteLLM.

https://t.co/hzutGUMDe7

[Google DeepMind @GoogleDeepMind]

Meet Gemini 4 Hydrogen: built for rapid coding, multimodal synthesis, and everyday workflows.

It finishes tasks 2.5x faster than 3.8 Flash and surpasses Argon on the @ArtificialAnlys Intelligence Index.

[LiteLLM (YC W23) @LiteLLM]

Rate-limit responses now tell you which exact limit caused a 429 error or triggered a fallback.

Here is how it works in two scenarios:
1 - No fallback available: LiteLLM returns a 429 with rate_limited_by set to the exact rule that blocked it (e.g., model_per_end_user).

2 - Fallback used: If Opus hits its limit and routes to Sol, LiteLLM adds response headers showing that 1 fallback was attempted and the exact rate limit that triggered it.

PRs #51228 #51138

[LiteLLM (YC W23) @LiteLLM]

Gemini Decision 1 now supported on the LiteLLM Gateway.

- Send a request to /v1/decisions with gemini/gemini-decision-1 and your questions
- A question can be yes/no, a choice or a score
- The model returns a probability for each answer
- Costs $0.10 per 1M input tokens

Read more: https://t.co/9uFW8RB4r9

[Sundar Pichai @sundarpichai]

Today we’re launching the Gemini Decisions API, powered by our new Gemini Decision 1 model in @GoogleAIStudio.

Instead of generating text token by token, it’s built for closed-set choices scoring up to 1,000 questions across text, images, audio, video + PDFs in a single pass. A few details:

- 3 decision types: yes/no, multiple choice (up to 255 options) + rubric scoring
- $0.10 / 1M input tokens with $0 billed for output tokens (~5x cheaper than Flash-Lite)
- leads on several benchmark evals with lowest calibration error

Google teams are already building with it, below is a look at Stitch Rapid designing a full product flow in seconds (1x speed)

[Moe Khalil @Mokhalil01]

Introducing performance alerts in @LiteLLM

LiteLLM now flags spend spikes, traffic spikes, higher error rates, and slower responses across your teams, users, keys, and models

Read more below ↓

[LiteLLM (YC W23) @LiteLLM]

All routing in LiteLLM now flows through a Rust core!

Check out this deep dive by Nate Armstrong (@armstrong_nate) on how it was achieved.

[Nate Armstrong @armstrong_nate]

See how LiteLLM rewrote the core routing logic in Rust! This post is a technical deep dive that goes over the refactor before the rewrite and an in-depth explanation of how we are leveraging differential testing and agents to avoid regressions.

https://t.co/BBy4kh6G3I

[LiteLLM (YC W23) @LiteLLM]

Skills Gateway - Share your favorite agent skills with your team!

[Tin Chi Lo @tinchil]

We've revamped our Skills Gateway

That useful skill you've built for yourself? Now you can upload it to the @LiteLLM Skills Hub and share it with the rest of your team

Read more in our blog post: https://t.co/FsrI5pxNDp

[LiteLLM (YC W23) @LiteLLM]

Local AI Cluster serving models behind LiteLLM 🫶

[Mehdi @MehdiHacks]

Hybrid local AI cluster: 4 DGX Sparks + 2 Mac Studios.
In total: 1 TB unified memory.

[LiteLLM (YC W23) @LiteLLM]

Local open-source models 🤝 LiteLLM gateway

[Tous.une.idée.en.Tech / @Pi3r2Dev]

L'atelier ; une seule illustration.

Chaque objet a une zone cliquable taillée à sa silhouette (clip-path: path()) et un calque découpé dans la même image, qui s'éclaire au survol.
Maj allume tous les objets.

La vidéo est un script Playwright : il lit les sous-titres dans les aria-description des objets.

Derrière : NestJS 11 sur Fastify, Prisma sur Postgres avec pgvector, files BullMQ, workers Python avec Celery et CrewAI (que je vais changer pour langchain) pour les agents à plusieurs étapes.

Crawl, raisonnement journalier, veille concurrentiel, crm de prospection auto alimenté, base vectoriel, multi posting social, crawl "interdit" par plugin chrome ou script console, mise en journal papier des FAV, Bots telegram, ingestion de contenu.

L'app ne connaît aucun nom de modèle, seulement des alias (default, analyse-think, reasoning, vision, embedding) servis par une passerelle LiteLLM.
default tourne sur un Qwen3-VL-8B sur une RTX 3070 ; le 27B tourne à la demande sur des RTX 3080, il est parfois éteint pour faire de l'image ou de la vidéo à la place avec des script on/off de flotte.

Les alias ont des replis : un autre modèle local, puis un cloud gratuit, et pour les embeddings tout un ART.
La réflexion se règle par alias dans la passerelle, jamais dans l'app.

Les alias de la synthèse quotidienne ne pointent que vers des GPU locaux, sans repli. Un script du harnais vérifie que le code de la synthèse n'appelle aucun autre alias. Il ne lit pas la config de la passerelle, qui vit dans un autre dépôt : cette moitié-là tient seulement par convention.

Les règles d'archi sont des scripts qui sortent en 1, avec un cliquet sur la dette existante. L'un d'eux vérifie l'ordre des migrations Prisma : une migration mal nommée et hop la boum.

Une (énorme) partie du code s'écrit en boucle agentique. La boucle ne commit jamais : je relis le diff, et le harnais (tsc, Jest, Vitest, les vérificateurs) doit être vert avant que je commit.

Vends moi des modèles open source ici ⬇️⬇️

[Maryam Miradi, PhD @MaryamMiradi]

🛠️🧭 "How We Solved Agent Building" got me digging.

If you are building AI Agents for real business workflows, you have probably struggled with:

agent handoffs, lost context, complex orchestration, failed tool calls, and agents that work in demos but struggle with real user questions.

I watched a talk on AI Engineer about how Vercel tried several agent architectures before a simple file system beat their multi-agent system.

So I pulled together their 6 stages and the open-source tools worth exploring at each one.

Here are 19 GitHub repos worth knowing for AI Agent building in 2026:

》𝗦𝘁𝗮𝗴𝗲 𝟭: 𝗢𝗻𝗲 𝗕𝗶𝗴 𝗣𝗿𝗼𝗺𝗽𝘁

✸ Vercel AI SDK → unified model access
✸ LiteLLM → multi-provider LLM gateway
✸ Instructor → structured LLM outputs

→ Test whether the LLM can generate valid SQL.

》𝗦𝘁𝗮𝗴𝗲 𝟮: 𝗠𝘂𝗹𝘁𝗶-𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺

✸ LangGraph → stateful agent orchestration
✸ CrewAI → role-based agent collaboration
✸ Google ADK → multi-agent conversations
✸ OpenAI Agents SDK → agents + handoffs

→ Specialized agents, but context gets lost in handoffs.

》𝗦𝘁𝗮𝗴𝗲 𝟯: 𝗢𝗻𝗲 𝗦𝘁𝗮𝘁𝗲𝗳𝘂𝗹 𝗔𝗴𝗲𝗻𝘁

✸ PydanticAI → typed agents + dependencies
✸ Deep Agents → planning + persistent context
✸ Mastra → workflows + memory + tools

→ One agent can plan, execute, and recover from errors.

》𝗦𝘁𝗮𝗴𝗲 𝟰: 𝗙𝗶𝗹𝗲 𝗦𝘆𝘀𝘁𝗲𝗺 𝗔𝗴𝗲𝗻𝘁

✸ Claude Code → filesystem-based coding agent
✸ just-bash → Bash + virtual filesystem
✸ Vercel Sandbox → isolated code execution

→ File system + Bash roughly doubled the eval score.

》𝗦𝘁𝗮𝗴𝗲 𝟱: 𝗥𝗲𝘂𝘀𝗮𝗯𝗹𝗲 𝗔𝗴𝗲𝗻𝘁 𝗦𝗸𝗶𝗹𝗹𝘀

✸ Anthropic Skills → reusable agent instructions
✸ Skills CLI → discover + install skills
✸ Vercel Agent Skills → domain-specific skills

→ Reuse proven workflows instead of starting from scratch.

》𝗦𝘁𝗮𝗴𝗲 𝟲: 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗔𝗴𝗲𝗻𝘁 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸

✸ Eve → filesystem-first agent framework
✸ Workflow DevKit → durable execution
✸ OpenTelemetry → tracing + observability
→ Turn files, tools, and skills into deployable agents.

And this is the architectural progression:

Big Prompt
→ Multi-Agent System
→ Stateful Agent
→ File System + Bash
→ Reusable Skills
→ Production Runtime

𝗣𝗹𝗲𝗮𝘀𝗲 𝗥𝗲𝗺𝗲𝗺𝗯𝗲𝗿:

Sometimes better context, familiar tools, and a simple file system can outperform an elaborate multi-agent workflow. The trick is knowing when to add complexity and when to remove it.

I may benchmark LangGraph vs Google ADK vs PydanticAI vs Deep Agents vs Eve next, using Insurance Claims, Wind Turbine Maintenance, or Legal Contracts.

[LiteLLM (YC W23) @LiteLLM]

Per-model TPM and RPM limits now apply to fallback models.

Here is how per-model limits work with fallbacks in these two scenarios:
1 - Opus fails and the key is under its sol limit: LiteLLM sends the request to sol and attributes the tokens to sol, the model that served it.

2 - Opus fails and the key is at its sol limit: LiteLLM does not send the request to sol, and the caller receives the opus error.

PRs #51082 #50994

[LiteLLM (YC W23) @LiteLLM]

Day 0 support for Gemini 4 Carbon on LiteLLM!

$2.00 / $10.00 per 1M tokens until November 30

Use it with LiteLLM here: https://t.co/w5XFwxE62i

[Google @Google]

Gemini 4 Carbon is now available.

It’s our strongest model, built to dig deeper, handle long-horizon tasks, and multi-step reasoning.

If Argon is great for complex tasks, Carbon is the model you bring in for your hardest problems.

[LiteLLM (YC W23) @LiteLLM]

Using the LiteLLM Gateway, Codex now lists every model in /models, including Claude and Gemini models.

[LiteLLM (YC W23) @LiteLLM]

Day 0 Support for Claude Fable 5.2 on LiteLLM

- $8 / $40 per 1M tokens, 20% below Fable 5.1
- Cache reads $0.20 per 1M
- 1M context, 128K output, default effort high

Details on how to use Claude Fable 5.2 with LiteLLM: https://t.co/b4eIJgYYzs

[Claude @claudeai]

Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges.

They’re capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.

[LiteLLM (YC W23) @LiteLLM]

@MongoDB Thanks! Details on the Voyage AI models supported on LiteLLM: https://t.co/VeF39Nm3Mk

[LiteLLM (YC W23) @LiteLLM]

@Blum_OG @Shruti_0810 Yes, that's built in. One could easily configure fallbacks and switch providers automatically. https://t.co/Jm1ZX8PNF3

[LiteLLM (YC W23) @LiteLLM]

The Model Hub within the Gateway Dashboard now shows pricing for every model deployment: cached, long prompt, flex, priority, and batch. This makes it easy to find all pricing info in one place.

[Moe Khalil @Mokhalil01]

@LiteLLM now supports -latest model aliases.

Generally, every new model release in a family is smarter, faster, and cheaper than its predecessor.

With -latest, you pick a model family once, and LiteLLM handles updates to newer versions while you keep the same model name.

[LiteLLM (YC W23) @LiteLLM]

LiteLLM this week: routing fully in Rust, -latest aliases, Skills Hub and more.

Join our live office hours to hear from the team - Fridays at noon Pacific.

0:00 Routing is fully in Rust
0:29 New models
0:53 Codex model picker
2:04 -latest model aliases
2:32 Aria and the AI Hub
3:07 Skills Hub

[LiteLLM (YC W23) @LiteLLM]

Also merged by our team:

- Claude only lists temperature and top_p when the model accepts them (#50144)
- Missing Claude limits now fall back to family defaults (#49815)
- GitHub Copilot sign-in works with your own GitHub app (#50116)

[LiteLLM (YC W23) @LiteLLM]

LiteLLM now supports -latest aliases. Pick a model family once. We update the model behind the alias as newer versions arrive, so you can keep the same name in your code.

Details here: https://t.co/02Xfiuf1VY