Meta published a paper that might end the current LLM era.
They proved that we can kill the "tokenizer" forever and it actually makes the models stronger.
Every AI you use today (ChatGPT, Claude, Gemini) reads text the exact same way. It chops your words into arbitrary chunks called tokens.
It relies on a massive, bloated vocabulary of around 100,000 specific chunks just to understand a sentence.
We assumed this was the only way to build AI. We treated it as the fundamental speed limit of language models.
Meta proved we were completely wrong.
Their new paper, "Breaking the Token Ceiling," ditches tokens entirely.
Instead, they forced the AI to process raw bytes. A vocabulary of just 256 values.
The results rewrite the rules of model efficiency.
By training on pure bytes and distilling the knowledge, these tiny models didn't just survive. They dominated.
Meta's distilled 1B-parameter byte models asymptotically surpassed state-of-the-art token models like Llama 3.2-1B and Gemma-3-1B across downstream tasks.
They use a fraction of the vocabulary. They require less data. They take up less storage.
It forces the model to actually learn the fundamental relationships between raw data instead of relying on pre-packaged text chunks.
This is a massive shift for anyone building, training, or deploying AI.
We’ve spent the last five years obsessively optimizing how we chop up text to feed these models.
It turns out, the bottleneck wasn't the intelligence of the model.
It was the tokenizer.
リズと青い鳥 第四乐章有很长的ユーフォニアム Carry..
按照设定来说应该是久美子担当。也就是四舍五入久美子也Solo过了🥹
Re:【Sound! Euphonium】Liz and the Blue Bird Full Performance (All Movemen... https://t.co/TJHyTgyZB5 来自 @YouTube
My friend Alex just got on twitter. He has a PhD in category theory from Northeastern, and he's brilliant. He's also working for a math neolab many of you know and love. I *highly* recommend following Alex, and if you're category-curious, you should attend his class! I will!!