GPT-6 Astra, 2026: more than 100,000 GPUs.
GPT-1, 2018: 8 GPUs, 30 days.
12,500x in eight years, both numbers from OpenAI directly.
the paper that introduced the transformer trained on 8 P100s for three and a half days.
this is.. insane
@ChaseLochmiller@OpenAI GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years.
AGI has arrived. Congratulations @OpenAI team.
400K GPUs coming online next.