Announcing the @AutobenchAI Router (and some bad news for @AnthropicAI and @OpenAI).
Yes, today we are releasing an LLM router that really saves more than 90% of your AI bill (with $1 free credits to try).
On Monday our Router Benchmark showed you that current routers don't route at all (the blue dots in the graph). That was the good news for AI Labs. Today comes the bad news: we built a router that actually works (the green dots).
Setting our router to "Quality" or "Balanced", it beats the top reference model (Gpt-5.6 Sol) at respectively 12% and 8% of Sol's bill. Setting it at "Cost", it offers even better quality and savings than Luna.
Try it now on our user-facing service Botscanner and claim your $1 free credits (links below). That's the equivalent of about 270 hard prompts (only 13 if you were using Gpt-5.6 Sol).
1/7👇
Today, if you say that maidan 2014 was a staged US coup, there are only two options:
- either you’re an idiot (a useful idiot)
- or you are an accomplice
And I know for sure that Elon Musk is not an idiot
Elon Musk on Ukraine:
It's true that from an invasion standpoint, Russia is the aggressor, but it is also true that we Americans staged a coup to overthrow the democratically elected pro-Russian president of Ukraine.
Without U.S. State Department and other agency help, this would not have happened.
That is obviously an offensive action.
@_fracapuano No, sinceramente, Le Cun lo vedrai mettere like a molti post che gli piacciono. Non era per diminuirti. E, cmq, io ne ho solo uno. Tu due😉
Ecco, mancavano all’appello gli “esperti“ nostrani di Taiwan e semiconduttori. I primi a voler vendere chip alla Cina erano proprio TSMC e i taiwanesi, proprio perché l’esclusione della Cina dalla supply chain dei semiconduttori è visto dal Partito Cominista Cinese come il principale rischio geopolitico per la Cina (con tutti i rischi che ne conseguono all’indipendenza di Taiwan). Huang è celebrato dai taiwanesi come un vero eroe nazionale, il figlio più illustre della nazione. E i taiwanesi sanno che da questa defenestrazione non verrà assolutamente nulla di buono per la loro sicurezza.
Cosa fa una presunta “esperta” di politica internazionale quando spara una sequela di balle stratosferiche su Jensen Huang, sulla competizione USA-Cina nei semiconduttori, sugli interessi taiwanesi nella filiera, dimostrando di non avere la più pallida idea di ciò di cui scrive? Ti insulta, poi blocca i commenti al post e anche allo stesso retweet che si è fatta del suo stesso post. Un capolavoro di gaslighting.
E non so se tutto ciò faccia più pena o piu ridere.
@ossogatto@19marcello65@flancini Che titolo? Ho fatto contestazioni fattuali, facilmente verificabili o falsificabili. Non ho ricevuto alcuna contro argomentazione o smentita fattuale, solo insulti. E, pertanto, posso fare a meno dei “lavori” di questa “esperta”
“Cafone”, “ma come si permette?”, “fisico”, “bagno di umiltà”, “fattoide”. L’esperta di Taiwan può continuare quanto vuole con gli insulti, il fatto è che ha detto una quantità di fesserie stratosferiche (Monopoli assoluti, chip avanzati, ecc. ecc.) e ha, addirittura, tacciato di “tradimento” Jensen Huang, sposando a pieno la narrazione costruita ad hoc da Rubio, pentagono e pezzi vari della Silcon Valley. La cacciata di Huang da Nvidia è l’evento più catastrofico nella storia geopolitica di Taiwan. Avrà conseguenze funeste perché le opzioni per la Cina ora sono solo due: o si ricostruisce da sola la filiera dei semiconduttori avanzati (e ciao ciao Taiwan), oppure si prende direttamente la filiera con la forza. Ma capisco che il ragionamento non è abbastanza “umile” per “esperte” così ben “informate”…
Chiudo: sostenere che Jensen Huang sia un traditore (cosa che nessuno si azzarda a fare a Taiwan) è vergognoso, scellerato e abominevole.
Balle. Balle su balle. L’unica cosa che si capisce qui è che lei non sa neanche cosa siano questi fantomatici “chip avanzati” e neppure di come operino i player taiwanesi e di quali siano i loro interessi.
Fintanto che fab, packaging, e euv sono fuori dalla Cina, TSMC (e il governo taiwanesi) sono perfettamente felici di vendere capacità produttiva anche a 2nm, cosa che, infatti, TSMC stava negoziando con i produttori cinesi prima che l’amministrazione Biden imponesse le restrizioni. E in questo Huang e TSMC andavano perfettamente a braccetto nel cercare di rilassare le restrizioni USA. Dire che poi Huang è un traditore di Taiwan è pura fantascienza. Huang è forse una delle persone più popolari a Taiwan, ma dopo la sua defenestrazione c’è stata una vera e propria esplosione di simpatia nei suoi confronti (e di rabbia per la sua cacciata).
L’unica domanda qui è perché lei si mette a sparare tutte queste balle?
While the US and China battle it out on GPUs (the Jensen Huang drama being only one in the long saga) consensus on the high-end semiconductor gap is estimated at 3-5 years (it was 5 years just 2 years ago). The broad US ecosystem still holds significant advantages, especially in scale of production, but, in the end, the real hard gap is euv lithography where no short-cutting is possible. That’s an asset locked primarily by ASML, a European company. It’s time for Europe to start leveraging this asset and play hard ball with the US. Europe is critically dependent on the US for its own security (at least for the next 5-10 years). The deal is clear: unwavering commitment by the US to NATO (not words, but real deployments in the most critical areas: Nordics, Baltics, Poland) in exchange for the euv advantage. Else, since export restrictions are so fashionable these days, access to the technology should be put under strong scrutiny (to say the least).
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
A few weeks ago we said LLM routers don't route. Then @OpenRouter shipped a new one.
On 6 November they launched a rebuilt Auto Router. So our team @AutobenchAI ran it on the exact questions from our October router benchmark. Not what you would expect. At its top setting its Best Answering Ratio went from 79.1% to 65.2%. A random pick scores 64.7%.
We also tested two new routers, @coniferbuild (a winner) and @daridotdev.
Here details of what we found:
1. The new OpenRouter Auto Router at its best setting scores about the same as picking a model at random (65.2% vs 64.7%). Its other three settings score below random. It agrees with the old router on only 1–12% of picks. Its top picks are @Nvidia Nemotron Ultra, @MiniMax_AI M3 and @deepseek_ai V4 Flash. At its highest-quality setting it never picks @OpenAI GPT-5.6 Luna, the best-value model on our board.
2. Conifer does route. It beats a random pick from its own models by almost 15 points. It still loses to always calling Luna (lower quality, higher bill), but it is by far the top performer among all routers we tested (except for our own Autobench router).
3. Dari scores 79.1%, the same as the old OpenRouter Auto Router. But it only chooses among 8 strong models. A random pick from those 8 already scores 73.9%.
So the October result stands: no commercial router we tested beats calling one mid-priced model... Well, no router except our own. The AutoBench Router at Balanced scores 87.8%, above always calling GPT-5.6 Sol (87.5%), at 8% of Sol's bill. Try it and get $1 free credits.
Links are in the replies.
1/3👇
Want a router that actually routes? Try ours and get $1 free credits.
Pick a setting, not a model. Balanced beats always calling GPT-5.6 Sol at 8% of Sol's bill. Cost beats always calling Luna on quality and price.
$1 free credit, about 270 prompts on Cost: https://t.co/8nMuve2grO
Or point your agent at https://t.co/4O0JA5YGJY
https://t.co/ilBoRZbLnW
3/3 end 🧵
Announcing the @AutobenchAI Router (and some bad news for @AnthropicAI and @OpenAI).
Yes, today we are releasing an LLM router that really saves more than 90% of your AI bill (with $1 free credits to try).
On Monday our Router Benchmark showed you that current routers don't route at all (the blue dots in the graph). That was the good news for AI Labs. Today comes the bad news: we built a router that actually works (the green dots).
Setting our router to "Quality" or "Balanced", it beats the top reference model (Gpt-5.6 Sol) at respectively 12% and 8% of Sol's bill. Setting it at "Cost", it offers even better quality and savings than Luna.
Try it now on our user-facing service Botscanner and claim your $1 free credits (links below). That's the equivalent of about 270 hard prompts (only 13 if you were using Gpt-5.6 Sol).
1/7👇
Want to know more about how we test routers?
We use long, dynamically generated, open-ended questions, answered by 20 models and scored by a jury of 10 AI models. A router names a model and we look up that model's answer. The percentage is how often the pick was as good as the best answers. To be fair to OpenRouter: the new Auto Router only accepts 15 of our 20 models, so it ran on those 15. Their own benchmarks show gains. This is one benchmark, on long questions, not the last word.
Everything is reproducible and public.
Full write-up: https://t.co/o0JFRTlhTa
Data (Wave 3 is in the wave3 folder): https://t.co/2SSpfU3hGF
October benchmark: https://t.co/Qfyv8Ezy5F
OpenRouter's announcement: https://t.co/CI7b0zvLv7
X benchmark announcement: https://t.co/sDUVp6Gmdh
2/3
Do LLM routers work? I have some good news for @AnthropicAI and @OpenAI: LLM routers suck.
Routers are systems that, given a prompt, choose among a set of LLMs which is the best to answer. The goal is to identify the model that guarantees a high quality response at the lowest cost possible. The problem they promise to solve is simple: why call Gpt Astra on a prompt that a low-cost open source model like @deepseek_ai can answer just as well? A nightmare scenario for the big labs like OpenAi and Anthropic.
Well, we applied the @AutobenchAI benchmarking methodology to 7 major routers. Result: always calling a mid-tier model like GPT-5.6 Luna beats every router we tested on both quality and cost. No exceptions.
Check out the quality-vs-cost Pareto board. The light shaded zone is the bare minimum prize: better than a random pick and cheaper than a random pick.
Luna is already sitting in it. Every router we tested is outside it.
It's even worse. The routers that look cheap just collapsed onto one flash model and landed on the random line. The ones that look strong just bought an expensive model.
1/14 👇