Day 3 of Learning #NamasteAI
Season 1 | Episode 4
🔐 The Secret Language of LLMs
Today, I explored how LLMs process human language and convert it into something a computer can work with.
@akshaymarch7 📜 A Long Prompt Doesn't Mean a Better Result
Adding more information does not automatically make an AI response better.
Clear, relevant, and well-structured context is often more useful than simply making a prompt longer.
Learning AI, one episode at a time.
@akshaymarch7 📚 Vocabulary and Token ID
A vocabulary is the collection of tokens that a tokenizer knows.
Each token is mapped to a unique numerical Token ID, which is what the model actually processes.
@akshaymarch7 🧩 Why do models use Subword Tokenization?
Subword tokenization can represent both common and uncommon words efficiently.
Instead of needing a separate token for every possible word, a word can be broken into smaller meaningful pieces.
One episode at one time.
Day 2 of Learning #NamasteAI
Does ChatGPT Know or Does It Guess?
I explored some fundamental concepts behind Search Engines, LLMs, ChatGPT, Hallucinations, and RAG.
📚 RAG — Retrieval-Augmented Generation
RAG combines information retrieval with LLM generation.
It retrieves relevant information from an external knowledge source and provides that context to the LLM to help generate a more grounded response.
Thanks @akshaymarch7