SpaceX has almost finished writing V1.0 of an in-house AI training stack in C that exact-maps to 220k GB300s with 800G NICs, making heavy use of pipeline parallelism and getting as close to bare metal as possible.
The potential speed improvement vs JAX for large training runs is over an order of magnitude.
Next will be writing the inference stack in C for simultaneous high-speed RL across a large block of GB300s.
(We do use a little C++ tbh, but not much)
Elon Musk on the physics of truth:
“Truth is not an easy thing
But you can aspire to the truth. You can try to get as close to the truth as possible with minimum error while acknowledging that there will be some error in what you’re saying
This is how physics works. You don’t say you’re absolutely certain about something, but a lot of things are extremely likely
Aspiring to the truth is very important”
Her bayram arefesi, İzmir Çamdibi Müzisyenler Kahvesi önünde gerçekleşen bir ritüel. Tüm Balkan göçmenleri kahve önünde buluşarak şarkı türkü söyleyip eğleniyor. ❤️