Jay Mody

programmer

[swyx 🇸🇬 @swyx]

This is still the best* introduction to GPTs I have ever read.

https://t.co/WNVKnQVMtn

Fills in all the gaps between Attention is All You Need right up to coding every single line of a working GPT2 implementation yourself. Even ends with explanations of finetuning for classification, summarization, instruction tuning, and PEFT. Bravo @jaykmody and all who take the time to write these things down in fully commented python rather than cryptic latex.

*best WRITTEN introduction. we all know who has the best video intro to GPTs…

[Andrej Karpathy @karpathy]

Late to the party but "GPT in 60 Lines of NumPy" / picoGPT is nicely done: https://t.co/lSEgz8qf0M
- good supporting links/pointers
- flexes some of the benefits of JAX: 1) trivial to port numpy -> jax.numpy, 2) get gradients, 3) batch with jax.vmap
- inferences gpt-2 checkpoints

[Jay Mody @jaykmody]

@karpathy @akshay_pachaar @karpathy Thank you!! I learn from the best 😉