Indic LLM

Documenting and tracking the exploding Indic LM ecosystem.
Project Bhaskar | @Adaptiv_me.

[Indic LLM @Hardik_Sen_16]

@SarvamAI : You don't need billions of params to punch above your weight.

22 Indian languages
Apache 2.0
Write & chat ready

If you were to use Sarvam translate - what languages would you experiment with?

#IndicLM #AIForBharat @ai4bharat

[Indic LLM @Hardik_Sen_16]

This is so exciting!!

Indic LLM model Navarasa being showcased at Google I/O is huge!!

To know what is going in the world of Indic LMs: visit https://t.co/sfqrDLAWut

[Ravi Theja @ravithejads]

🔥 Navarasa at Google I/O 🔥

🎉 I'm super thrilled to share that our work (with @ramsri_goutham) - Navarasa has been showcased at Google I/O!

🌐 Navarasa is a Gemma-finetuned Indic LLM covering 15 Indic languages (+ English).

It was great to see indicchat developed by @SatpalPatawat in the showcase. 👏

📺 If you missed it at Google I/O, you can check it out here: https://t.co/NIUIxqoOtC

[Indic LLM @Hardik_Sen_16]

@ravithejads @ramsri_goutham Congratulations @ravithejads !! We are monitoring the progress of Indic LLM on our website: https://t.co/sfqrDLAWut and would love your inputs on Navarasa model!

[Indic LLM @Hardik_Sen_16]

@akshay_pachaar The explantion about KV catching and stuff is just amazing!!

[Audel @Audel_Rouhi]

@akshay_pachaar I created a new type of database that includes native hierarchical KV caching for local LLMs. https://t.co/rZ8fsMEafr

[Indic LLM @Hardik_Sen_16]

This is so well explained @akshay_pachaar !! Would love to discuss with you more on this in terms of Indic LLM and as we are monitoring their progress - your kind of insights will be really helpful!

[Akshay 🚀 @akshay_pachaar]

You're in an ML Engineer interview at Anthropic.

The interviewer asks:

"Our model generates 100 tokens in 42 seconds. How do you make it 5x faster?"

You: "I'll optimize the model architecture and use a better GPU."

Interview over.

Here's what you missed:

The real bottleneck isn't compute. It's redundant computation.

Without KV caching, your model recalculates the same attention keys and values for every single token generation.

That's why a 9-second inference becomes 42 seconds. You're wasting 80% of your time on repeated calculations.

The fundamental issue:

(refer image below as you read ahead)

LLM token generation is autoregressive:

- Generate token 1 from the prompt
- Generate token 2 from prompt + token 1
- Generate token 3 from prompt + token 1 + token 2

At each step, you're reprocessing ALL previous tokens through attention.

Token 50? You've computed attention for token 1 fifty times.

The reality of attention mechanism:

For each token, the transformer computes:

- Query (Q) from current token
- Key (K) from all previous tokens
- Value (V) from all previous tokens

Then: Attention(Q, K, V) = softmax(QK^T)V

Problem: K and V for previous tokens never change. You're recalculating identical matrices every single step.

How KV caching solves this:

Instead of recomputing K and V matrices:

- Cache them after first computation
- Reuse cached values for subsequent tokens
- Only compute K and V for the new token

Without KV caching (token 50):
- Compute Q, K, V for all 50 tokens → O(n²)

With KV caching (token 50):
- Load cached K, V for tokens 1-49
- Compute Q, K, V only for token 50 → O(n)

You've eliminated quadratic redundancy.

So what's the tradeoff:

While KV caching makes the inference faster, it also takes up a lot of memory, so there is always a tradeoff between speed and memory.

Why your first token always takes longer:

KV caching speeds up inference by computing the prompt's KV cache before generating tokens.

This is exactly why ChatGPT takes longer to generate the first token than the rest.

First token: Computing KV cache for entire prompt

Remaining tokens: Just loading cached KVs + computing new token

----
Everything above is how KV caching works inside a single request.

Running it in production is a different problem. Caches break on document reordering, and a single GPU throws away roughly 15 TB of reusable cache per day.

I wrote an article that picks up exactly where this post ends: how a new open-source architecture manages KV cache at production scale, with 14x faster time-to-first-token to show for it.

The article is quoted below.

[Akshay 🚀 @akshay_pachaar]

You're in an ML Engineer interview at Anthropic.

The interviewer asks:

"Our model generates 100 tokens in 42 seconds. How do you make it 5x faster?"

You: "I'll optimize the model architecture and use a better GPU."

Interview over.

Here's what you missed:

The real bottleneck isn't compute. It's redundant computation.

Without KV caching, your model recalculates the same attention keys and values for every single token generation.

That's why a 9-second inference becomes 42 seconds. You're wasting 80% of your time on repeated calculations.

The fundamental issue:

(refer image below as you read ahead)

LLM token generation is autoregressive:

- Generate token 1 from the prompt
- Generate token 2 from prompt + token 1
- Generate token 3 from prompt + token 1 + token 2

At each step, you're reprocessing ALL previous tokens through attention.

Token 50? You've computed attention for token 1 fifty times.

The reality of attention mechanism:

For each token, the transformer computes:

- Query (Q) from current token
- Key (K) from all previous tokens
- Value (V) from all previous tokens

Then: Attention(Q, K, V) = softmax(QK^T)V

Problem: K and V for previous tokens never change. You're recalculating identical matrices every single step.

How KV caching solves this:

Instead of recomputing K and V matrices:

- Cache them after first computation
- Reuse cached values for subsequent tokens
- Only compute K and V for the new token

Without KV caching (token 50):
- Compute Q, K, V for all 50 tokens → O(n²)

With KV caching (token 50):
- Load cached K, V for tokens 1-49
- Compute Q, K, V only for token 50 → O(n)

You've eliminated quadratic redundancy.

So what's the tradeoff:

While KV caching makes the inference faster, it also takes up a lot of memory, so there is always a tradeoff between speed and memory.

Why your first token always takes longer:

KV caching speeds up inference by computing the prompt's KV cache before generating tokens.

This is exactly why ChatGPT takes longer to generate the first token than the rest.

First token: Computing KV cache for entire prompt

Remaining tokens: Just loading cached KVs + computing new token

----
Everything above is how KV caching works inside a single request.

Running it in production is a different problem. Caches break on document reordering, and a single GPU throws away roughly 15 TB of reusable cache per day.

I wrote an article that picks up exactly where this post ends: how a new open-source architecture manages KV cache at production scale, with 14x faster time-to-first-token to show for it.

The article is quoted below.

[Indic LLM @Hardik_Sen_16]

@AndrewYNg That is correct! People need to change their workflow to get better results! The AI literacy gap just keeps increasing and very few notices!

Read more on it in our article: https://t.co/1RZvxnWRdb

[Indic LLM @Hardik_Sen_16]

The way Indian System is still condoning this is shocking!! We are still in taking decision and the loss is too much!

People talk about taxes, insurance, carious other things ton cure economical crisis but this is the base

[Sarah @SarahGairola]

A century of India's brightest mind, leaving.
Just finished going through every decade of it, 1924 to 2024. Starting to share what I found this week.

[Indic LLM @Hardik_Sen_16]

India’s AI can’t just be a western model with a translation layer. To fix the bias against low-resource languages, we need native tokenizers, cleaner data pipelines, and hyper-local context.
See how we’re planing to build the future of native Indic LLMs: https://t.co/SOXTsNr9qe

[Indic LLM @Hardik_Sen_16]

Weekend poll 🗳️

Which Indian language do you wish AI understood better?

Vote + tell us why 👇

#Languagemodel #indiangovernment #marathi #tamil #IndicLM #Polls #vote2026 #sarvamai #tamilllama #mahamarathi7b

[Geetanjali S @plaingeets]

75 Years Ago, India Installed a Bug in Its System. We're Still Running the Same Code. https://t.co/3zMrHRDs95 via @Substack

[Indic LLM @Hardik_Sen_16]

New Indic models drop almost every month.

We built the site so adding one model updates every page automatically — catalogue, languages, use-cases.

One source. Always current.

Explore:https://t.co/o9UlQMZCJh

[Indic LLM @Hardik_Sen_16]

'Indic LLM' sounds technical. It isn't.

1. LLM = software that learns from lots of text so it can read, write & answer.
2. Indic LLM = one that does it in Indian languages.

That's it.
See every Indic LLM in one place → https://t.co/Gvb5XFsIHf

#IndicLLM #AI #IndianLanguages

[Indic LLM @Hardik_Sen_16]

The generative AI boom is incredible, but there’s a massive gap. To bridge the gap I am excited to announce the launch https://t.co/Gvb5XFsIHf — the ultimate discovery hub & tracking platform for Indic Large Language Models with Project Bhaskar!

[Indic LLM @Hardik_Sen_16]

@TheVixhal This is a fantastic and highly actionable breakdown. When analysing the models powering the Indic AI ecosystem for Project Bhaskar, it always comes back to these fundamentals.