holy f*ck, Karpathy put an entire AI degree on GitHub for free
7 repos, 306,741 stars, and in the right order they take you from backprop to agents that run research overnight
each repo is the next layer of the one before
1. micrograd -> how a neural net actually learns
backprop in about 100 lines of Python
↳ https://t.co/sZdt2ZhTs5
2. nn-zero-to-hero -> the video course
his lecture notebooks, from one neuron to GPT
↳ https://t.co/3rINVTCeY7
3. minbpe -> how text turns into tokens
the tokenizer every LLM starts with, in clean minimal code
↳ https://t.co/eeC6ySUzES
4. nanoGPT -> your own GPT
the simplest repo to train and fine-tune one
↳ https://t.co/pw8JPMvFmf
5. llm.c -> the same training with nothing hidden
raw C and CUDA, no PyTorch underneath
↳ https://t.co/neU4gdN71w
6. nanochat -> a full chat model end to end
GPT-2 level training cost ~$43,000 in 2019
here it's about $48
↳ https://t.co/MBcyMrTtRd
7. autoresearch -> agents that do the research
an agent edits the training code, trains for 5 minutes, keeps what improved and repeats all night
↳ https://t.co/EoyCfdPydo
train one GPT from scratch and you become the person who gets handed the AI project, along with the raise that comes with it
the classroom half of the same degree, what Stanford actually teaches about AI, is below ↓