We added Gemini 4 Argon to our AutoResearchExam Pareto curve. It is 2-3 times cheaper compared to the other frontier models for almost the same ML engineering performance. Congratulations to the GDM team.
@kartikb753@madiator@bespokelabsai 50 pp is plain accuracy, so both count the same. When you split by type: picks a model that can’t serve the request →
Nimble-v4-max 0%
Nimble-v4 17%
Jev 1.13 57%
All our misses for Nimble-v4-max picked a model that qualified but cost more.
Today, we are announcing Nimble-v4 and Nimble-v4-max which are pushing the frontier of decision models, outperforming Jev and other variants.
On the Decision Index public scores, nimble-v4-max and nimble-v4 score +10 and +6 against Jev 1.13, making nimble-v4-max the #1 model.
Beyond benchmarks, we have done a lot of internal evals, and see meaningful improvements: 43% fewer mistakes than Jev, 50pp improvement on model routing decisions, being great at coding use cases, fact checking etc.
The models are available via our API service.
Go build nimbly: https://t.co/nFnE4dn7OQ
Today, we are announcing Nimble-v4 and Nimble-v4-max which are pushing the frontier of decision models, outperforming Jev and other variants.
On the Decision Index public scores, nimble-v4-max and nimble-v4 score +10 and +6 against Jev 1.13, making nimble-v4-max the #1 model.
Beyond benchmarks, we have done a lot of internal evals, and see meaningful improvements: 43% fewer mistakes than Jev, 50pp improvement on model routing decisions, being great at coding use cases, fact checking etc.
The models are available via our API service.
Go build nimbly: https://t.co/nFnE4dn7OQ
I AM OFFICIALLY RESTARTING THE $1,000 TO $1,000,000 SEPTEMBER $SPY CHALLENGE NEXT MONDAY! 💸
I’M GOING TO RESTART AND LET EVERYONE FOLLOW MY EXACT TRADES FOR COMPLETELY FREE IN A PRIVATE X GROUP CHAT! 🦅
LIKE, REPOST, & COMMENT “$1K” TO BE ADDED! ❤️🔥
YOU MUST BE FOLLOWING ME TO JOIN! ☢️