Andrej Karpathy’s 1-hour Stanford lecture on AI engineering is one of the best explanations I’ve seen of how AI systems actually work.
The progression is simple:
10% → LLM
30% → Prompt
50% → Agent
70% → Loop
100% → Graph
The key takeaway:
AI engineering isn’t just about writing better prompts.
It’s about building systems around models — giving them context, memory, tools, feedback loops, and data flows.
“Delete everything, keep Graph.”
Definitely worth watching if you’re building with AI agents.
Watch → Bookmark →
ICYMI: We rolled out our upgraded Notebook experience to 100% of Pro users. You should feel this improvement across the entire flow, but just in case, here's a tldr;
💬 Upgraded chat: smarter, more thoughtful interactions
📊 New outputs: create charts, PDFs, spreadsheets, images & more
🔎 Agentic research: start with just a loose idea and some questions, Notebook will handle the rest
As always, we really appreciate your feedback. Let us know what you think!
TinyLoRA: LoRA scaled down to 1 parameter.
Researchers from Meta, Cornell, and CMU just dropped a banger.
They turned an 8B parameter model into a math and reasoning powerhouse by tweaking just 13 of those parameters.
That's 26 bytes and takes up less storage than this sentence.
The model hit 91% accuracy on GSM8K, up from 76% before the tweak.
The method is called TinyLoRA, and it pushes low-rank adaptation to its absolute extreme.
Some quick background on LoRA first:
When you finetune a large model, you're updating billions of parameters. LoRA showed you can instead learn a small low-rank update on top of frozen weights, bringing that down to millions.
LoRA-XS compressed this even further by leveraging the internal structure of the weight matrices, bringing it down to tens of thousands.
TinyLoRA goes all the way down to one.
Here's how:
> Instead of learning a matrix-sized update, learn a tiny vector that gets expanded into a full weight update through a fixed projection. only the tiny vector is trainable.
> Tie this vector across all modules and layers so the entire model shares the same tiny set of trainable parameters.
> With full weight tying, the entire model update collapses to as few as one trainable parameter.
I have shared a really nice illustration to explain TinyLoRA in the next tweet.
But the real insight is not the architecture. it's that this only works with reinforcement learning.
When they tried SFT with the same tiny updates, performance barely moved. SFT at 13 parameters hits 83%. RL hits 91%. to match RL performance, SFT needs 100x to 1000x more parameters.
This is because SFT forces the model to memorize full demonstration trajectories, treating every token as equally important. RL only passes back a sparse reward signal, and through resampling, the useful signal accumulates while the noise cancels out.
This means RL is not teaching the model new knowledge. it's making a precise, tiny adjustment to unlock reasoning the model already has.
One more surprising finding: as model size grows, the number of parameters needed to reach peak performance shrinks. this suggests trillion-scale models might be tunable for specific tasks with literally a handful of bytes.
Find the paper and TinyLoRA visual in the next tweet.
If you found it insightful, reshare it with your network.
Find me → @_avichawla
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. https://t.co/bl5W7pVBNM
Drag-and-drop UI to build AI agent workflows!
Sim is a 100% open-source alternative to n8n (Apache 2.0 licence)
Get started here: https://t.co/w5xr9KBkmW
GitHub repo: https://t.co/prWxUcfJfL