๐จIlya Sutskever finally confirmed
> scaling LLMs at the pre-training stage plateaued
> the compute is scaling but data isnโt and new or synthetic data isnโt moving the needle
Whatโs next
> same as human brain, stopped growing in size but humanity kept advancing, the agents and tools on top of LLMs will fuel the progress
> sequence to sequence learning
> agentic behavior
> teach self awareness
Think of it as the โiPhoneโ, which kept getting bigger and more useful from hardware point, but plateaued and the while focused shifted to applications.
2025 will be the year of Agents!
> @Replit for coding
> @seobotai for content
> @crewAIInc for the rest