Today we're releasing Agora: the first ever pretraining stack that allows non-collocated consumer GPUs to be competitive with centralized clusters
Agora is 15x faster than Megatron-LM in this setting and is only 1.5x less efficient in terms of tokens per unit compute than TorchTitan on H100s, despite running on devices that have no NVLink or InfiniBand support.
The first Protocol Learning workshop in Rio today!
A new field is emerging: decentralized training of foundation models. Because the world’s intelligence belongs in open, collectively owned systems.
Thanks for joining @oguzer90@niclane7@m_ryabinin@namhoonlee09@itsmaddox_j
From fiber cuts to alleged missile strikes on datacenters, digital infrastructure is now part of the battlespace.
That’s why the space domain matters. Resilient connectivity and compute can’t depend on a single geography.
Data centers, fiber, satellites — this is all national security infrastructure.
I would like to host an initiation dinner for select new Miami residents who I’ve never met.
Absolutely no Brickell scene, no god forsaken Major Food Group restaurant. We are doing real Miami shit.
Please tag in your favorite new residents and I will respond with details accordingly.
'13. Labs will not be allowed to “escape” to neutral jurisdictions.
We are about to see a 10x in state power simply due to the fact that AI requires centralised, physical datacenters and these can be controlled in a way diffuse human knowledge work cannot.
@nic_carter@TheStalwart Most Americans can’t afford very basic healthcare and already have a deep distrust for pharmaceutical and insurance companies.
Drug discovery likely doesn’t even crack the average persons top 30 concerns wrt AI.