this is actually huge. sota models could directly provide hidden states to smaller models, allowing a 4B parameter model capable of running on edge devices to perform decoding.
it's a new way to do fast and cheap inference.
shut out to people who first made it.
will agent talk in latent space in the future? probably.
meet @mostik_ai!
what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can.
everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?
we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched.
how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance.
we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst
WIRED has the first external account of the company and the work: https://t.co/tP8nItCsDl
full writeup, the setup, and all the numbers: https://t.co/C9NZ5vtV1V
A 753B model reads the problem: A 4B model writes the answer.
Mostik reports that its latent bridge lets the 4B model close half the performance gap to the 753B model
The bridged pair uses 2.5x less compute than a score-matched mid-sized model
A fascinating alternative to the text-based handoffs used by most multi-model systems today. Incredible!
This architecture is really interesting
Most model-to-model systems today communicate through text, so one model has to turn its reasoning into tokens before the next model can use it.
Mostik puts a trained bridge between the two models instead, passing hidden states directly from the sender to the receiver.
The models stay frozen, while the bridge learns how to translate between their internal representations.
That lets a much larger model handle the reasoning while a smaller model handles the answer, which is a pretty interesting approach to model composition.
The argument that CoT monitoring is already weak is correct and it does not automatically license moving reasoning off the text surface. Both things can be true.
To their credit they seem to know that.
Mostik is a cracked team of math PhDs that invented a new way of scaling LLMs by letting models directly share their internal compute states instead of communicating through text.
In just 4 months of lock in, they got to #1 on ARC-AGI-3.
Congrats on coming out of stealth @aimalysheva!
The open vs frontier debate gets one thing wrong. You don’t always need the biggest model to get the job done.
If you can separate the reasoning from the model itself, the whole game changes.
This is one of the coolest breakthroughs I've seen with LLMs.
A 753B model "thinks" about the answer, and a 4B model "writes" the answer. The communication between the models happens in latent space, resulting in performance nearly as good as the big model, but 20x faster.⬇️