BREAKING: Meta's Llama 4 Maverick just hit #2 overall - becoming the 4th org to break 1400+ on Arena!🔥
Highlights:
- #1 open model, surpassing DeepSeek
- Tied #1 in Hard Prompts, Coding, Math, Creative Writing
- Huge leap over Llama 3 405B: 1268 → 1417
- #5 under style control
Huge congrats to @AIatMeta — and another big win for open-source! 👏 More analysis below⬇️
Our friends @Meta are taking part in the 8VC Llama Stack Innovation Challenge!
First challenge: 3/7-4/19. Use Llama Stack to build desktop, field, and edge AI applications that run locally for privacy and performance. Join now:
https://t.co/B5ncyoB9UD
🚀 We're Hiring! 🚀
Our LLaMA Speech team at Meta GenAI is hiring for full-time Research Scientists based in the US! We are particularly interested in senior researchers who have demonstrated deep expertise in audio & speech LLMs. If you are interested, please DM me with your CV.
This works great for those use cases where the answer to those tests is deterministic and not subjective
That’s why it’s so good for coding and math- there’s only one single correct answer
However, there’s a lot of use cases for which that evaluation is much more subjective
One of the ways DeepSeek was able to train its models for less...
Have you ever taken a test before, you got an answer right, and your professor marked it wrong? Then you go back to the professor and have to argue with them and it's a pain in the butt?
Well, if there is only one answer, and it's a very simple answer, then you can say, "write that answer in this box and you either get it right or wrong." There is no arguing.
What the DeepSeek team did was, rather than having human beings check the model outputs and say yes or no, they developed code to say: here’s the box, output the answer and then check it.
If it's correct, we have the answer. If not, we don't.
No need to involve a human. Completely automated.