Still feels surreal to have been on stage with one of the greatest CEO's of our time Dr Lisa Su where she explicitly called out GPU MODE and the work we did to enable the world's first $100K competitive kernel competition
I never in a thousand years would have imagined that a humble reading group would morph into a machine that can generate more kernel data than what exists on all of Github combined and that the best members of our community could beat the best baselines by the best human experts
Thank you GPU MODE, my favorite corner of the internet
New modeling study finds that if everyone in the United Kingdom had worn N95 or FFP2 masks from the start of the pandemic, the rate of COVID-19 transmission would have been 9 times lower.
Scalify
scale propagation for efficient low-precision LLM training
Low-precision formats such as float8 have been introduced in machine learning accelerated hardware to improve computational efficiency for large language models training and inference. Nevertheless, adoption
It’s impossible not to cry every day. This disease has taken everything. For a year and a half I’ve lived in a nightmare and made no progress getting out. Severe #LongCovid is something no one should have to endure.
I had so many dreams.
I wanted my own science show on Netflix.
I wanted to start a family with Kyle.
I wanted to scuba in Australia.
Now, I just want to walk again.
I want to have a real conversation.
I want to read a book again.
I want to listen to music and see my family.
Super long 🧵 incoming. I really enjoy looking at the full stack of computing - maximizing intelligence per picojoule.
Been wanting to do a far-too-in-depth overview of the ENTIRE thing - from manufacturing a GPU to training a GPT :D
Here we go!
Hi everyone yes, I left OpenAI yesterday. First of all nothing "happened" and it’s not a result of any particular event, issue or drama (but please keep the conspiracy theories coming as they are highly entertaining :)). Actually, being at OpenAI over the last ~year has been really great - the team is really strong, the people are wonderful, and the roadmap is very exciting, and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you who’ve followed me for a while may have a sense for what that might look like ;) Cheers
8-bit floating point is the next tour de force to train and deploy LLMs.
Today we present our new @graphcoreai preprint:
📢 Training and inference of large language models
using 8-bit floating point 📢
https://t.co/rIB3jpdEAo
Details on the 🧵[1/7]