A 39 year old Slovak-Canadian researcher just paused his AI school to join the company that's racing OpenAI for the future of intelligence.
His name is Andrej Karpathy. And his career is one of the strangest in modern AI.
Karpathy was born in Bratislava in 1986, in what was then Czechoslovakia. His family moved to Toronto when he was 15. He did his bachelor's in computer science and physics at the University of Toronto, where he sat in on Geoffrey Hinton's classes and reading groups long before Hinton became a household name. He did his master's at UBC. Then he went to Stanford for his PhD under Fei-Fei Li, the woman who built ImageNet.
In 2015, while still finishing his PhD, he designed and taught Stanford's first deep learning course. It was called CS231n. It started with 150 students. Two years later it had 750. The lecture videos went on YouTube for free and became the standard reference for an entire generation of AI engineers. TIME magazine later reported that those Stanford lectures alone have over 800,000 views.
That same year, Sam Altman, Elon Musk, and Ilya Sutskever started a research lab called OpenAI. Karpathy was one of its founding members.
He stayed at OpenAI for two years. Then in June 2017, Elon Musk hired him to run Tesla Autopilot. He spent five years there as Senior Director of AI, leading the computer vision team behind every Tesla on the road.
In February 2023 he went back to OpenAI. He quit again in February 2024.
And then he did something nobody in his position had ever done.
He turned his YouTube channel into a school.
His "Neural Networks: Zero to Hero" series walks through how to build a GPT model from scratch in plain code, math, and English. The flagship video, "Let's build GPT: from scratch, in code, spelled out," is just under two hours long and has millions of views. People at every major AI lab cite it as how they actually understood transformers.
In July 2024 he announced Eureka Labs. The pitch was an AI-native school. Human teachers paired with AI teaching assistants, scaling one great teacher to a million students. The first course was LLM101n, an undergraduate program that would teach anyone to train their own ChatGPT from scratch.
Then in February 2025 he coined the phrase "vibe coding" in a single throwaway tweet. The phrase became Collins Dictionary's Word of the Year. It now has its own Wikipedia article.
Last week, on May 19, 2026, he announced he is joining Anthropic.
Eureka Labs is on pause. He said the next few years at the frontier of large language models will be especially formative, and he wanted to get back to research. At Anthropic he will lead a pretraining group focused on using Claude to accelerate Claude itself.
The man who left two of the most powerful AI labs on Earth to teach the world for free just walked back into one of them.
The videos are still up. All of them. Free. Forever.
And he still answers questions in the comments.
We implemented @karpathy 's MicroGPT fully on FPGA fabric.
No GPU.
No PyTorch.
No CPU inference loop.
Just a transformer burned into hardware, generating 50,000+ tokens/sec.
The model is small, but the idea is not: inference does not have to live only in software 👇
ANDREJ KARPATHY COULD HAVE CHARGED $2,000 FOR THIS COURSE.
He put it on YouTube.
The full training stack. Tokenization. Neural network internals. Hallucinations. Tool use. Reinforcement learning. RLHF. DeepSeek. AlphaGo.
3 hours of the most comprehensive LLM education that exists anywhere at any price.
Not how to use the tools.
How the entire system was built from the ground up and why it behaves the way it does.
The engineers who understand this build things the ones who only use the tools cannot even conceive of.
The gap between those two groups is not 3 hours.
It is everything those 3 hours quietly unlock for the rest of your career.
Everyone is fine-tuning LLMs.
Almost nobody understands what is actually being updated inside the model.
Here are 5 techniques that change how you think about model adaptation, and what each one is actually doing to the weights:
1./ LoRA - Learn the update, not the weights
The pretrained weight W is frozen. Completely untouched.
Instead of updating W directly, two small matrices are trained =>
A ∈ ℝʳˣᵈ and B ∈ ℝᵈˣʳ, where r ≪ d
The weight update is: ΔW = BA Effective weight: W' = W + BA
The entire adaptation happens in a tiny low-rank space. W never changes.
2./ LoRA-FA - What if we freeze even more?
Same structure as LoRA. One change.
A is frozen alongside W. Only B is trained. Effective weight: W' = W + BA (A is fixed)
Half the trainable matrices of LoRA. Same core idea. Fewer parameters.
3./ VeRA - What if the matrices don't need to be learned at all?
This is where it gets interesting.
A and B are both frozen, and randomly initialized. What gets trained are just two tiny scaling vectors =>
b ∈ ℝʳ and d ∈ ℝʳ
Instead of learning the low-rank matrices themselves, VeRA keeps them frozen and learns small scaling vectors that modulate their contribution.
Initialization => b = 0, d = 1
You're not learning matrices. You're learning how to scale them.
One of the most parameter-efficient techniques on this list.
4./ Delta-LoRA - What if W itself learns from the low-rank updates?
This one is fundamentally different.
Unlike standard LoRA, the base weight W is not fully frozen. It is updated through low-rank delta propagation at every step =>
W^(t+1) = W^t + c(B_(t+1)A_(t+1) − B_t A_t)
Where c is a scaling factor.
A and B are trainable. W evolves, but guided entirely by low-rank changes.
5./ LoRA+ - Same structure. Smarter learning rates.
Identical to LoRA, freeze W, train A and B.
One change => B is assigned a larger learning rate than A. ��_B > η_A
A ← A − η_A · ∂J/∂A B ← B − η_B · ∂J/∂B
A small optimization change that can make LoRA training more effective.
The core idea running through all five:
You do not always need full fine-tuning to adapt a model.
LoRA updates two matrices.
LoRA-FA updates one.
LoRA+ updates two at different speeds.
Delta-LoRA lets W evolve - guided by low-rank deltas. VeRA updates two vectors.
Same goal. Five different answers to the same question:
=> What is the minimum we actually need to learn?
That is the core idea behind parameter-efficient fine-tuning.
And now you know what is actually happening inside the model.
A guy in Karachi rebuilt GPT-4 in one Jupyter notebook.
OpenAI spent over $100 million to train the real one. He put the entire recipe on GitHub for free.
His README still says "I am looking for a PhD position in AI."
It's called Train LLM From Scratch. A working guide that walks you through building your own 2-billion-parameter language model on a single GPU.
OpenAI vs this repo:
- Training cost: $100M+ → Single A100 or RTX 4090 (you can rent for $1/hr)
- Code access: Closed → Open, MIT license
- Data: Secret → The Pile (open dataset, 825GB)
- Walkthrough: None → Every line of code explained, top to bottom
- Output quality: GPT-4 → A small model that writes broken English (but it's yours)
The whole thing fits in one notebook. No paid course. No paywall. No "Pro" tier.
What you actually learn:
→ How a transformer works, end to end
→ How to download and tokenize the Pile dataset
→ How to build multi-head attention from scratch in PyTorch
→ How to train on a single GPU without running out of memory
→ How to generate text from your trained model
→ How to scale from 13 million parameters to 2 billion
774 stars. 135 forks. MIT license. The full theory paper-to-code in one place.
One honest note: this is a learning repo, not a production model. Your output will be small and rough. But you will understand exactly how GPT-4 works after reading it.
Fareed Khan built this from Karachi, Pakistan. He has 1,780 GitHub followers. He's still looking for a PhD position. The recipe to billion-dollar AI is sitting on his profile, free.
This is what open AI was supposed to mean.
(Link in the comments)
New article: a visual tour of recent LLM architecture advances, from Gemma 4 to DeepSeek V4.
I focus on long-context efficiency tweaks like KV sharing, per-layer embeddings, layer-wise attention budgets, compressed attention, and mHC.
Link: https://t.co/KO81y3kTH7
🚨ALERT🚨Our system has detected 7.2M suspicious transactions involving @THORChain router across $BTC, $Ethereum, $BSC and $Base networks.
Stolen funds include: $USDT. $USDC, $WBTC, $DAI, $THOR, $LUSD, $XRUNE, $GUSD, $AAVE, $LINK $FOX etc.
All stolen funds are swapped to $ETH and consolidated at 0xd477b69551f49C0519F9B18c55030676138890Bd
If you wish to safeguard yourself against such scams, please contact us to arrange a demo of our solution at https://t.co/PdJL6YkwOF
#CyversAlert
This morning, THORChain was drained of roughly $10.8m
Node operators have freezed the network for nearly 13 hours. The full analysis isn't out yet, but according to @jpthor, this could be a MPC exploit.
ECDSA and TSS is hard. THORChain's vaults rely on TSS, a flavor of MPC where a quorum of nodes jointly produces a signature without ever reconstructing the private key. Clean for Schnorr or EdDSA; painful for ECDSA, which Bitcoin and Ethereum require. That's why we saw plenty of protocol attempts (Lindell17, GG18, GG20, CMP, CGGMP21, DKLS, KU23...), each patching flaws in the previous one.
GG20 has a track record. THORChain's TSS uses GG20, on a fork of Binance's tss-lib. GG20 has shipped two well-publicized critical bugs: CVE-2023-33241 and TSSHOCK. CGGMP21, now cggmp24, are the latest protocols, but GG20 is still widely deployed.
I often hear a misconception when I hear about MPC setup: "The key is split across many nodes, so any single co-signer doesn't really matter".
In every published GG18/GG20 attack, one malicious or compromised co-signer is enough to extract everyone else's shard and reconstruct the full key.
AI changes the threat model. Compromising a full software node, complex Go stack, exposed P2P, custom signing daemons, a churn protocol that admits new participants on a schedule, has always been difficult and acted as a barrier. With LLM-driven vulnerability discovery and exploit synthesis, the bar to compromise one of N validators is dropping fast.
Here, it's a plausible TSSHOCK-style playbook:
- compromise one operator
- wait for it to churn into an active Asgard vault
- send malformed proofs during keygen or signing
- reconstruct the key offline
- sweep in a single transaction
It's unclear yet if the attacker used a known-unpatched GG20 weakness, or a fresh cryptographic flaw.
But, in all cases, MPC and TSS are not a substitute for hardening every co-signer. They sit on top of co-signers that must each be treated as critical infrastructure, hardware-isolated enclaves, minimally exposed, continuously audited, and running protocol with security proofs.
While the investigation progresses, be careful in your interactions onchain. These TSS setup are used in various protocols.
This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc.
More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage:
1) raw text (hard/effortful to read)
2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default
3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default
...4,5,6,...
n) interactive neural videos/simulations
Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral https://t.co/z21CP5iQfu
There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen.
TLDR The input/output mind meld between humans and AIs is ongoing and there is a lot of work to do and significant progress to be made, way before jumping all the way into neuralink-esque BCIs and all that. For what's worth exploring at the current stage, hot tip try ask for HTML.