It is dangerously easy to build a neural network today without actually understanding how it works.
We live in an era of 'import torch'. You can train a model in three lines of code, but the moment you need to debug a collapsing loss function or a vanishing gradient, syntax won't save you. You need first principles.
I recently went through this notebook collection by Simon J.D. Prince, and it is the antidote to tutorial hell.
Instead of just showing you the code, it forces you to visualize the mechanics:
1./ The Math => It builds the intuition for shallow networks and regions before adding complexity.
2./ The Optimization => It doesn't just use an optimizer; it compares Line Search, SGD, and Adam so you see why they behave differently.
3./ The Modern Stack => It connects the dots from basic backpropagation all the way to Self-Attention and Graph Neural Networks.
Move from running code to engineering systems => this is a goldmine.
As an AI Engineer. Please learn:
- Harness engineering, not just prompt engineering
- Prompt caching vs. semantic caching tradeoffs
- KV cache management at scale
- Speculative decoding vs quantization
- Structured output failures & fallback chains
- Evals (LLM-as-judge + human evals)
- Cost attribution per feature, not just per model
- Agent guardrails & loop budgets
- LLM observability as a first-class discipline
- Model routing & graceful fallback logic
- Knowing when to fine-tune vs. in-context learning