After a short nap-when i wake up, itโs a whole new szn. ๐
Thank-you @X Family for supporting ๐ซถ
@LinkedIn -ย https://t.co/YzHa8xppUR
@github -ย https://t.co/i2EvVTB9O0
@kaggle -ย https://t.co/oJSj4HKuZc
@LeetCode -ย https://t.co/vdIjIIqIlm
@TensorTonic -https://t.co/gEq77SC58V
6/6
6 days ago I started a "backprop from scratch" project.
Full notebook + theory + interview cheat sheet ๐
GitHub link - https://t.co/3Jadyyqc0w
Kaggle - https://t.co/aptzW1UCSY
5/6
My favorite part of this project: looking inside the network.
These are the first-layer weights, reshaped as 28ร28 images. Each hidden neuron became a little stroke or edge detector.
Nobody programmed them. They emerged from nothing but gradient descent on 60,000 digits.
4/6
SGD vs Adam, both written from scratch. Same seed, same init, same batch order
SGD: 97.65% (10 s)
Adam: 97.55% (32 s)
Adam hit 97% two epochs sooner (epoch 3 vs 5) and reached 3ร lower training loss. But the test accuracy is a tie. The 0.1% gap is about 10 images out of 10K
3/6
How do I know my hand-written gradients are right?
I don't trust myself.
W1 โ 2.5e-11 โ
b1 โ 2.8e-11 โ
W2 โ 2.2e-11 โ
b2 โ 1.8e-11 โ
broke it on purpose: forgot to divide by batch size. Error jumped to 0.78 โ. Caught instantly.
ML bugs fail silently -build detectors
2/6
The entire backward pass of a neural-network layer:
dW = Xแต @ dY
db = dY.sum(axis=0)
dX = dY @ Wแต
My favorite debugging trick is the shape check. XแตยทdY is (in,N)(N,out) = (in,out), exactly W's shape. Wrong shape means wrong derivation
1/6
I trained a neural network on MNIST with no PyTorch and no autograd.
Every gradient: derived by hand, coded in NumPy.
Result: 97.65% test accuracy.
Hi guys Back here ! -- will be posting my progress with some new projects ,,, First I will start with some warm-up projects and then moving to a Production-Grade End-to-End Capstone projects , See you soon guys !!!!!!!!
Hi @X Family
@Flipkart GRiD 8.0 update: cleared R1 โ , R2 didn't go my way ๐
1 Q solved clean, rest got wrecked by compilation errors + bad time management
Lesson learnt, prep mode: ON for next time ๐ฅ
#GRiD8#flipkart