It's fascinating once you understand that the speed of light is actually constant–no matter what speed you're going in. Travelling close to the speed of light, light still seems to move at the the speed of light away from you. What makes the possible is the time slowing down.
I don’t understand how this makes sense, isn’t the whole point of ML frameworks to make prototyping, and then hand over the computation via hardware acceleration?
SpaceX has almost finished writing V1.0 of an in-house AI training stack in C that exact-maps to 220k GB300s with 800G NICs, making heavy use of pipeline parallelism and getting as close to bare metal as possible.
The potential speed improvement vs JAX for large training runs is over an order of magnitude.
@ludwigABAP Same for me, SAS flight from Stockholm to Paris a few days ago had Starlink, only realised because of a small brochure with the flight info