Jev vs Decision-machine-1 a model that was already doing what Jev does but nobody heard about.
Founders don't hang out on x, go figure.
Took Matteo's game & adapt it to let the models play.
DM-1 crushed Jev on all my test games💀 and is cheaper
https://t.co/2MCUAV2AI0
Spent some this morning putting two AI classifiers head to head: @typesafeai vs @millisecondsai. I used the same 2,000 news headlines and labeled categories to see which classifier would yield better results. Thread 🧵
Thanks for the detailed work here, super appreciated!
On latency and speed, some insights:
We have three zones for now:
- enam (montreal)
- wnam (oregon)
- weur (netherlands)
Both our API gateways and GPUs live very close in these zones. The gateway sits entirely at the closest Cloudflare datacenter from the GPU to minimise round trips. Some tech insights here: https://t.co/0mIWq24Dhk
For accuracy, definitely our next concern! We come from a "business paperwork" background, and our models are fine tuned on financial/business/legal-shaped documents.
Expanding to "other subjects" is our next milestone, and evals like yours will help us get there fast!
@debugsenpai if you're lining up Jev alternatives, worth adding https://t.co/k8Gcj3OLE2 😉 same "decision machine" idea, but it takes image input too (Jev doesn't). Demos: https://t.co/VM8X8fa1vG