𝒶𝓁𝒾𝓈

19 • seeking deep

[𝒶𝓁𝒾𝓈 @import_alis]

@Yaashix97 Excellent momentum Yashika!

[𝒶𝓁𝒾𝓈 @import_alis]

@deepzone_3 i'm currently referring directly to the research papers

[𝒶𝓁𝒾𝓈 @import_alis]

DeepSeek sparse attention breakdown: completed the formal analysis and indexer formulation for the Lightning Indexer layer.
@deepseek_ai

[𝒶𝓁𝒾𝓈 @import_alis]

@deepzone_3 i did mla after that. Now i've started sparse attention.

[𝒶𝓁𝒾𝓈 @import_alis]

@yffeng3920 Appreciate the breakdown. I'll revisit this after finishing the DSA paper.

[𝒶𝓁𝒾𝓈 @import_alis]

@divagr1925 I literally started with DSA today. Looking forward to following this series!

[𝒶𝓁𝒾𝓈 @import_alis]

started with dsa ( deepseek sparse attention)
and now the adrenaline is kicking in.

[𝒶𝓁𝒾𝓈 @import_alis]

Here's what i did :

>> implemented in PyTorch with only formula reference
>> wrote a blog

blog link -https://t.co/WoQFwqIrYj

[𝒶𝓁𝒾𝓈 @import_alis]

spent 16hrs doing Deepseek's V2 Multi-head Latent Attention.

[𝒶𝓁𝒾𝓈 @import_alis]

spent 16hrs doing Deepseek's V2 Multi-head Latent Attention.

[𝒶𝓁𝒾𝓈 @import_alis]

studied DeepSeek-V2's "𝙈𝙪𝙡𝙩𝙞-𝙝𝙚𝙖𝙙 𝙡𝙖𝙩𝙚𝙣𝙩 𝙖𝙩𝙩𝙚𝙣𝙩𝙞𝙤𝙣", step by step carefully and did the whole derivation by myself without referring back to the paper to check my understanding.

commendable work by the research team of @deepseek_ai truly.

[𝒶𝓁𝒾𝓈 @import_alis]

i have learnt KV-Cache and implemented it successfully in Pytorch

Here is my Blog Link - https://t.co/ppJfJDPEgw

Github repo link- https://t.co/Tz98ft7SJJ

[𝒶𝓁𝒾𝓈 @import_alis]

The ultimate cheat-code to success

[Pranav Shyam @recurseparadox]

Don’t get degrees man.
Just implement papers, tweet about it, write good blog posts, create nice repos. Biglabs will come and beg you to work for them.

[𝒶𝓁𝒾𝓈 @import_alis]

I'll be the kimi to your fable.

[𝒶𝓁𝒾𝓈 @import_alis]

spent the whole night writing this article. it taught me how to translate difficult concepts into simpler explanations.

link : https://t.co/D99m8w3wly

[𝒶𝓁𝒾𝓈 @import_alis]

implemented transformer in pytorch &
built an indepth understanding of transformer architechture

[𝒶𝓁𝒾𝓈 @import_alis]

made notes on Transformer

link : https://t.co/xpE1LTfgXA

[𝒶𝓁𝒾𝓈 @import_alis]

made notes on Transformer

link : https://t.co/xpE1LTfgXA

[𝒶𝓁𝒾𝓈 @import_alis]

implemented self-attention, multi-head attention & masked multi-head attention individually in pytorch

[𝒶𝓁𝒾𝓈 @import_alis]

implemented self-attention, multi-head attention & masked multi-head attention individually in pytorch

[𝒶𝓁𝒾𝓈 @import_alis]

studying transformers since 2 days. I'm done with embeddings, self-attention, multi-head attention, masked multi-head attention.

[𝒶𝓁𝒾𝓈 @import_alis]

studying transformers since 2 days. I'm done with embeddings, self-attention, multi-head attention, masked multi-head attention.

[𝒶𝓁𝒾𝓈 @import_alis]

Day 11 of MIT Deep Learning

Continued with Lecture -5

Insights:
Learnt what a GNN is
Two main goals of it- Node embedding and Graph embedding
How adjacency matrix is used as a input to a neural network

+ Practiced Mathematics of Neural Networks and Deep Learning (1hr)

[𝒶𝓁𝒾𝓈 @import_alis]

Day 10 of MIT Deep Learning

Completed Lecture -4

Insights:
> Learnt how a CNN works
> How Architectures enables us to generalize outside the training distribution
> How different types of architectures behave while image generation