rohit

[rohit @rohit_____04]

Moving on to Transformers

Locking in on this course for next few days...
Watched @karpathy's GPT from scratch , got solid understanding of self attention, though yet to implement the assignments , will work on those parallely

https://t.co/Nn7yzedPd2

[rohit @rohit_____04]

Started makemore!!!
@karpathy 🐐

[rohit @rohit_____04]

Day 10 : Deep Learning from scratch

“ pip install ronet”

Completed the library to implement neural nets✅
Successfully trained a model on MNIST dataset achieving 0.97 validation accuracy 🎯
Upload the package on PyPI

https://t.co/66sW0Z1BML
https://t.co/Ri56oR3tpo

[rohit @rohit_____04]

Day 9 : Deep Learning from Scratch

Added Batch training feature✅
Done with all the basic functionalities of a Neural Network ✅

Only robust testing remains now…..

[rohit @rohit_____04]

Day 8 : Deep Learning from Scratch

Finished Model class for training and prediction interface .✅

Got almost same results for a random regression task for my library and PyTorch so……. not a bad day👍

[rohit @rohit_____04]

Day 7 : Deep Learning from Scratch

Implemented L1 , L2 Regularization✅
Implemented Dropout Layer✅

[rohit @rohit_____04]

Source : https://t.co/xxUavcOsQ6

[rohit @rohit_____04]

Combined Softmax activation and Cross Entropy Loss
Fast and compact way to calculate gradients for softmax layer and cross entropy loss.

Reduces time for calculation by about 7 times.

Only one subtraction operation!!

Derivation:

[rohit @rohit_____04]

Combined softmax+crossentropy

Just one Subtraction!!!

[rohit @rohit_____04]

Gradients calculation for cross entropy loss:

- y_true/ y_pred

Requires one Division operation

[rohit @rohit_____04]

@karthikponna19 If not MacBook, any Linux distro on ‘Windows laptop’

[rohit @rohit_____04]

Day 6 : Deep Learning from scratch

Done and dusted with optimizers✅✅

Found a trick with reduces the computation of gradients for softmax layer by upto 7 times…. Planning on sharing the derivation tomorrow

Btw I’m currently following this book :

https://t.co/Dok63aXNqS

[rohit @rohit_____04]

@AvinashSingh_20 @karpathy ‘s videos are best if anyone wants to do deep learning and transformers from scratch

https://t.co/Z14QUk2xYv

I’m trying to build a deep learning api from scratch starting with basic neural nets

[rohit @rohit_____04]

The pen and paper version is the best way to learn imo, took less than 30mins to implement once I understood it on paper 📝

[rohit @rohit_____04]

Day 5 : Deep Learning from scratch

Thought I knew backprop until I had to implement the vectorized version😮‍💨

Spend the afternoon doing backprop for a batch of inputs on paper 😭

[rohit @rohit_____04]

Day 4 : Deep learning from scratch

Discovered a new source for deep learning from scratch ( beyond just neural networks) : Deep Learning from Scratch: Building with Python from First Principles https://t.co/abqQn9ntFs

Didn’t got much done today :(
Gave a cf contest ✅

[rohit @rohit_____04]

Day 3 : Deep learning from scratch

Switching from @karpathy ‘s version to numpy implementation

[rohit @rohit_____04]

Day 2 : Deep learning from scratch

Completed back propagation and implemented activation functions

Added api to build MLP

[rohit @rohit_____04]

@Loop_and_lift Hello!!!