60-70% of the world’s computers run on Linux. Yet Microsoft and Apple are the commercial leaders.
Wouldn’t be surprised if LLMs go the same way. We’ll still have Anthropic, OpenAI, and all the other new frontier labs...but open-source models will lead real-world adoption.
There is a world where your agent can know everything about you without the company that built it knowing anything about you.
I’d like to live in that world.
@omarsar0@subquadratic Agreed, most of those decisions are specific to your business so I'd train the model on your own agent's calls rather than rent.
On our contract benchmark the trained model came in at $1.03 per 1k questions, frontier flagship was $20.94.
Rather steep for picking an option 🤷♂️
Coinbase built this for their employees.
Overmind ships it for everyone.
Run /overmind finetune in:
• Claude Code
• Cursor
• Codex
Open source, and the model is yours.
60-70% of the world’s computers run on Linux. Yet Microsoft and Apple are the commercial leaders.
Wouldn’t be surprised if LLMs go the same way. We’ll still have Anthropic, OpenAI, and all the other new frontier labs...but open-source models will lead real-world adoption.
@naval models are the lab's moat. if your product rents the same one as everyone else, your moat is an API key. the useful asset is the data that teaches a model your particular task.
Abstraction is the core of innovation: taking something complex and making it simple, accessible, and usable.
That’s why we built Overmind. To abstract away the complexity of AI infrastructure and give everyone the tools to train and deploy their own models.
Overmind turns anyone into an AI lab.
• Builds a context graph from code and traces
• Curates training and eval datasets
• Evaluates prompts and models
• Trains smaller, specialized models
https://t.co/AEBRXmTE6M
Jev vs Clef settled one thing: a model anyone can rent is not a moat. The labels are.
A 4B model trained on 10k labelled banking queries made 47% fewer errors than Jev.
One LoRA run, no hyperparameter search, under an hour on Overmind.
Decision models had a big week. Clef, Strands, OpenAI's Decisions API, Jev already out.
They're great when you don't have labels yet. We fine-tuned a 4B on Banking77 anyway.
89.7% vs Jev's 80.4%. One LoRA, under an hour.
TLDR: the moat is your labelled data.
labs trained on vast amounts of third-party data without permission and now distillation is “stealing”. what exactly is being stolen? you can’t reconstruct the RL data or training process by prompting a model. this looks less like a moral objection and more like fear that someone else is catching up
scoring an untuned model gets you a long way, nice work. one place I'd test it: labels that come from your own definitions rather than general knowledge. we trained a 9B on yes/no relationship calls over 10,000 biomedical entity pairs and got 54.9% vs 14.1% for a frontier flagship. haven't run pg-jev or this approach on the same task, so no idea how it does. would be curious
some numbers from our own testing. contract clauses, 4,000+ questions across 100+ contracts. base model 80.4 F1, frontier flagship 82.0, the same base model trained on the task 91.6. quoting the clause word for word: frontier 8.5%, trained 59.6%. so model size wasn't what moved it, the task data was
the speed problem might be the wrong thing to chase with a bigger or faster general model. the local job is narrow: take a question, strip what identifies you, rewrite it. that's trainable. get a frontier model to rewrite a few thousand queries that way, check each for leaks, and train a much smaller model on the good ones. that replaces the skill file with something you can test, and it should be a lot faster than 20-30 TPS. catch is you need a proper set of leak cases to eval against, otherwise you're guessing