simrat hanspal

Exploring LLMs | Data scientist with a curious engineering mind

[simrat hanspal @simsimsandy]

My recent blog with @hasgeek - “Decoding Llama3” is out.

It’s a deep dive into the Llama3 model code released in April this year. This is a fun blog with a code-first approach.

https://t.co/ZdRHB8hW2S



[simrat hanspal @simsimsandy]

🫣 Softmax is unstable with very large and very small numbers.

🤓 Here is a simple illustration of how (x-max) makes softmax stable for use.

[simrat hanspal @simsimsandy]

What does it mean to have dropout in Attention computation?

Dropouts are used to prevent overfitting.
In case of attention, we drop some attention scores, which means that if the model learnt to attend to some token, it now has to focus on other related tokens.

#LLM #Attention

[simrat hanspal @simsimsandy]

I mean token to token embedding :’D

[simrat hanspal @simsimsandy]

Simple illustration of what token to word embedding conversion looks like.

[simrat hanspal @simsimsandy]

So, you use len(tokenizer)

Not sure why colab is not recognising len() :D

[simrat hanspal @simsimsandy]

The tokeniser lies about how many tokens it holds ;)

What the tokeniser returns is the size of the base vocabulary that it learnt during training. Everything after that are special tokens.

Special tokens are like metadata and help structure context.

[simrat hanspal @simsimsandy]

@Sam0kayy Balcony Lizards :/

[simrat hanspal @simsimsandy]

Trivial but worth a reminder use np.matmul for dot product instead of np. dot.

np. dot is meant to be a flexible function that will adjust according to the input shape, instead of raising an error.

Example np. dot(np.array([[1, 2], [3, 4]]), 10)

[simrat hanspal @simsimsandy]

@A_K_Nain Thank you so much for the summary👌
One question, why did they mask the prompt token in SFT?

[simrat hanspal @simsimsandy]

Budding entrepreneur 🥹
I purchased more than I planned.

[Zainab Bawa @zainabbawa]

And also assisting madame in coordinating logistics and order shipping the day after #fifthel
@jackerhack @_waabi_saabi_

[simrat hanspal @simsimsandy]

@fifthel @jackerhack I hope my pending orders arrive soon 😜

[simrat hanspal @simsimsandy]

@Anwxsha I waaaaaant !!

[simrat hanspal @simsimsandy]

Looking forward to it.

[anwesha @Anwxsha]

Tech x society enthusiasts, show up for The Fifth Elephant Annual Conference on 13th July!

I'll be hosting the session on Deploying AI in Key Sectors: Robust Risk Mitigation Strategies with @jnkhyati, @bargava, @simsimsandy & @fooobar

@fifthel @hasgeek @anthillin @zainabbawa

[Bengaluru Systems (fka Bengaluru Systems Meetup) @BengaluruSys]

First, @simsimsandy walked us through GPU architecture, optimizations, CUDA, and the challenges of running large ML models on GPUs, with a special look at the attention mechanism, KV-Cache optimizations, and PagedAttention!

[simrat hanspal @simsimsandy]

Thank you for the shoutout @TheOtherRaghav. It was a lovely event. https://t.co/uHrtrpCSAJ

[simrat hanspal @simsimsandy]

If you are into GenAI, @hasgeek is organizing a call today to build a community on #ResponsibleAI. Join for cross-learning.

🔗 Meeting Link: Register here to confirm your participation - https://t.co/R8WuiH76mS
🕰Time: 7 PM IST Friday, 28 June (tonight)

[Tune AI @Tunehq_ai]

🚀Join us in Chennai next week for our hands-on workshop: "Building AI Agents with RAG and Functions" 🤖✨

Limited seats available, so hurry and secure your spot! 🏃‍♂️💨

🔗 Register now: https://t.co/DYaWnmkREp

#AIWorkshop #ChennaiEvents #llm #genai

[simrat hanspal @simsimsandy]

Thank you for the call out @zainabbawa :)
Best wishes to all the speakers at FifthEl 2024, looking forward to networking in person.

[Zainab Bawa @zainabbawa]

@anscombes4tet @Aditi_ahj @fifthel .@simsimsandy introduced Bhumika Makwana @GalaxEyeSpace who will speak about multimodal fusion as the new game changer.
Reach out to Simrat for review and feedback on #nlp work, and for simplifying complex AI concepts. 3/5

[simrat hanspal @simsimsandy]

Really fun video on the basics - dot prd and inner prd.
Also, potentially a great resource on Quantum Mechanics #QuantumSense YT channel.

https://t.co/McXTOSDOed

Inner product is an important concept for Rotary Positional Embedding, which is used by #LLM like #Llama3 (#Llama).