Here's a common misconception about RAG!
When we talk about RAG, it's usually thought: index the doc → retrieve the same doc.
But indexing ≠ retrieval
So the data you index doesn't have to be the data you feed the LLM during generation.
Here are 4 smart ways to index data:
1) Chunk Indexing
- The most common approach.
- Split the doc into chunks, embed, and store them in a vector DB.
- At query time, the closest chunks are retrieved directly.
This is simple and effective, but large or noisy chunks can reduce precision.
2) Sub-chunk Indexing
- Take the original chunks and break them down further into sub-chunks.
- Index using these finer-grained pieces.
- Retrieval still gives you the larger chunk for context.
This helps when documents contain multiple concepts in one section, increasing the chances of matching queries accurately.
3) Query Indexing
- Instead of indexing the raw text, generate hypothetical questions that an LLM thinks the chunk can answer.
- Embed those questions and store them.
- During retrieval, real user queries naturally align better with these generated questions.
- A similar idea is also used in HyDE, but there, we match a hypothetical answer to the actual chunks.
This is great for QA-style systems, since it narrows the semantic gap between user queries and stored data.
4) Summary Indexing
- Use an LLM to summarize each chunk into a concise semantic representation.
- Index the summary instead of the raw text.
- Retrieval still returns the full chunk for context.
This is particularly effective for dense or structured data (like CSVs/tables) where embeddings of raw text aren’t meaningful.
👉 Over to you: What are some strategies that you commonly use for RAG indexing?
💠Over 6 Lakh Lost and Stolen Mobile Handsets recovered on DoT’s Sanchar Saathi Initiative
💠Advanced AI-powered platform demonstrates remarkable success in combating mobile theft
💠The success of the Sanchar Saathi App reinforces citizens' trust in digital governance
💠The upward Recovery Rates Demonstrate Platform's Growing Effectiveness and increased collaboration between DoT and State/ UTs police
💠Citizens are advised to report lost/ stolen mobile handsets on the Sanchar Saathi portal and Mobile App
Read here: https://t.co/FcAwDWYPkP
@PoliceShimla
Today, on 2nd October in Bemloi, 2 bikes were parked on the pedestrian walking path, blocking people’s movement.
Photo proof attached.
Kindly take strict action and ensure pedestrian paths stay clear.
I did my part, now it's your turn.
Here's my PvP guide for Grubbin Community Day this weekend! Charjabug is also learning Volt Switch and should see a lot more play in the future. Just for the weekend, I've given Charjabug search priority over Charizard as well. Have fun!
https://t.co/hoEHFxDTcl
Become a ghost for 6 months.
Make everything your fault.
Find the beast within you.
Throw yourself into pain.
Cut out all the excuses.
Go all in on yourself.
Train like a warrior.
Work like a robot.
Eat like a king.
Reject vices.
Transform.
Upgrade.
Create.
Thrive.
Win.
If you're not using AI, you're falling behind.
Here are 18 AI tools to future-proof yourself. Get more tools like this by signing up to Superhuman AI--the world’s biggest AI newsletter.