I made a video about what happens when you execute `c = a @ b` in PyTorch, from the code to silicon.
Hopefully it will help you appreciate that simple line of code more.
Made by Grok @bot. Enjoy!
We're releasing Terminal-Bench-Science: a benchmark for evaluating AI agents on research workflows across scientific domains.
An ongoing Stanford-led community effort, built by the team behind Terminal-Bench together with scientific domain experts at research institutions worldwide. v0.1 has 70 tasks. Claude Opus 5 solves only ~30%.
1/n 👇
If you are a founder, want to be a founder or understand the life of a founder, go read THE HARD THING ABOUT HARD THINGS immediately. It is a gem of a book.
What would I do if I had a one week off right now?
read https://t.co/3Nx4vds3Ro on
> kernels
> transformers
Try to reproduce the code snippets and fix bugs
In the process
> understand the topics
> get some contributor karma 💎
This is the most underrated AI course on the internet and its taught at Stanford and it costs zero dollars.
It's called CS336 and Stanford is teaching students to build a language model from absolute scratch.
Here's the actual curriculum:
→ Module 1: Tokenization, byte-pair encoding, and vocabulary design
→ Module 2: Transformer architecture written from raw PyTorch
→ Module 3: Distributed training and the pretraining loop
→ Module 4: Scaling laws and data curation
→ Module 5: Supervised fine-tuning and instruction following
→ Module 6: RLHF, DPO, and preference optimization
→ Module 7: Inference, KV caching, speculative decoding, and serving
→ Module 8: Evaluation, red teaming, and capability measurement
Every lecture is on YouTube. Every slide deck is downloadable. Every assignment is open.
Taught by Percy Liang and Tatsu Hashimoto. Two of the most cited researchers in the field of LLMs working today.
https://t.co/wSQeGd9oMG / https://t.co/ToEMzdddfH
If you finish this course, you understand how language models work at every layer from raw bytes to RLHF.
That is the difference between an AI engineer and an AI tourist.
Save this. Pick the first lecture tonight.
Pretraining Recurrent Networks without Recurrence by @akarshkumar0101 & @phillip_isola is a great paper!
It makes an end-run around the problems of RNNs via a transformer teacher to learn good predictive state representations & supervised learning of a memory transition function
Introducing celld: a self-hosted, distributed Durable Objects and Workers implementation
- celld = V8 + S3 + SQLite + LTX+ Tokio
- programmed with exactly the Cloudflare Workers/DO JavaScript APIs and config
- order of magnitude cheaper at scale
- writes are durable before they're acknowledged, RPO=0
- only depends on an S3 compatible bucket - no other infrastructure
- ~4 MB memory per cell instance overhead
- hibernated cells cost ~nothing, just their S3 storage
- open source, Apache 2 licensed
I stopped following the “learn everything from basics first” approach a long time ago. Most of the time, it just gives you the dopamine of progress without actually pushing you.
Put yourself directly into advanced, hard things. Struggle. Get stuck. Break things.
Then DFS + backtrack whenever you realize there’s a fundamental you’re missing. Learn that piece, then move forward again.
This has worked every single time for me.
You don’t need to know everything before starting hard things. Hard things will tell you exactly what you need to learn.
Underrated life advice: Become a beginner again. Join a run club. Learn the piano. Take up painting. Try woodworking. Plant a garden. Teach yourself to cook. Whatever. It slows you down. It’s a small rebellion in a world that wants you to do everything with some purpose in mind.