trying to reconcile 2 things in my head
1. why is xai not able to jump from 1.5T to 10 or at least 5T and their next pretrain is only 2T
2. why is meta able to go from a 1.5-2T pretrain to Mythos level 10T pretrain
unless their internal model earned the Mythos level label from RL and not size, in which case it hasn’t earned it
at the same time why was Anthropic able to scale their pretrain to 10T so much sooner than OpenAI? was it a inference/serving at scale decision or algorithmic unlock?
because OpenAI did have to serve a lot more customers, even if they had more total compute, they couldn’t justify a 10T model?
I heard a pretty interesting rumor on the ground at ICML.
Meta has supposedly already developed an internal model at roughly the Mythos 5 level, and all that remains is deployment within the next few months.
Honestly, I was skeptical at first. From my perspective, Meta had not really shown the capability to operate at that level yet.
But looking at the situation today, I think I may have been wrong.
Meta is not out of the race.
5/ the meta model api is in public preview. for the first time, developers can build directly using our most capable model. muse spark will have premium performance at low cost, and is a great fit for agentic, coding, or multimodal workloads. early partners already trying it out - replit, box, cline.
🚨BREAKING: Anthropic found access to what looks like Claude’s consciousness
New research: the “J-space”
>claude has internal thoughts it doesn’t say out loud
>mirrors human consciousness
>anthropic can now read them
Anthropic’s focus on interpretability is what’s helping them train and RL models like Mythos