Chinmay

ML Researcher @apnahq / @bluemachinesai

[Chinmay @chinmay_d__]

check the no speech probability output before trusting the tokens. don't take whisper's transcript on faith when its own confidence craters (3/3)

[Chinmay @chinmay_d__]

whisper inventing subtitles during actual silence isn't random, it's the training data leaking through 👇 (1/3)

[Chinmay @chinmay_d__]

Whisper’s training data included scraped captions with music, disclaimers, and outros over silence. When the audio signal is weak or silent, the decoder falls back on these learned priors and hallucinates text instead of predicting silence. (2/3)

[Chinmay @chinmay_d__]

the readme says "pip install -r requirements.txt" and works out of the box. it does NOT work out of the box

[Chinmay @chinmay_d__]

if you're building a streaming system, budget for RNN-T's extra decoder complexity upfront.

CTC still wins for cheap offline transcription where you can bolt on an external LM anyway (3/3)

[Chinmay @chinmay_d__]

CTC models are easy to train and still lose in production. streaming is why 👇 (1/3)

[Chinmay @chinmay_d__]

CTC assumes every output frame is independent given the encoder, it can't condition on its own last prediction, so it has zero language model baked in and stumbles on homophones.

RNN-T adds a predictor network that eats its own outputs, giving frame level context for free (2/3)

[Chinmay @chinmay_d__]

unpopular opinion: your eval set is vibes with extra steps if you've never read the raw transcripts yourself

[Chinmay @chinmay_d__]

the gap between "works in the notebook" and "works in prod" is where entire careers go to die

[Chinmay @chinmay_d__]

my swiggy order eta and my model's eta to convergence are both lying to me with the same straight face

[Chinmay @chinmay_d__]

If you're serving multiple tasks or languages, keep adapters unmerged and route dynamically at inference instead.

The minor latency gain you get from merging is rarely worth losing the flexibility to swap adapters on the fly. (3/3)

[Chinmay @chinmay_d__]

The LoRA finetuning decision people regret later:

Your adapter trains perfectly, so you merge it into the base model to speed up inference. Then you need to support a second language or task, and suddenly you're completely stuck. 👇 (1/3)

[Chinmay @chinmay_d__]

Because merging isn't just an optimization, it’s permanent.

Once that low rank update is fused directly into the base weights, there is no separate adapter left to swap out. You've essentially locked the model into a single task. (2/3)

[Chinmay @chinmay_d__]

The fix? Expand your target modules to include all linear layers.

Yes, it costs more trainable parameters. But for anything nontrivial, capturing those extra projections is usually the actual unlock, not some magic hyperparameter you thought you missed. (3/3)

[Chinmay @chinmay_d__]

A massive LoRA finetuning bottleneck most people hit without realizing it:

They only target q_proj and v_proj because it’s the tutorial default. Then the model plateaus, and it looks like a training or dataset issue, no matter how much you tweak things. 👇 (1/3)

[Chinmay @chinmay_d__]

But it’s not a training issue. It’s an architecture ceiling.

If your adapter never touches k_proj, o_proj, or the MLP layers, it literally lacks the capacity to represent complex tasks properly, no matter how high you crank the rank or learning rate. (2/3)

[Chinmay @chinmay_d__]

Next up for Iteration 2:
1. Tightening tool precision via better negative constraints & edge case data curation.
2. Improving Hindi number understanding (HNTA) to beat baseline limits on complex digit sequences and ordinals. (4/5)

[Chinmay @chinmay_d__]

The base Gemma 4 26B is an incredible model, but it still struggles with strict Hindi instruction adherence and complex tool calling in Hinglish.
So, I built Chemma 26B.
Just finished evaluating the iteration 1 checkpoints.
The deltas on complex tool calling are wild: (1/4)

[Chinmay @chinmay_d__]

Context & structured outputs are clicking, but tool precision (48.89%) and dialog tracking (37.5%) need optimization.

User retention took a slight hit (-10%), likely due to friction during multiturn testing. Still plenty of room to improve. (3/4)