Srini Rajagopal

VP engineering Mercor

[Srini Rajagopal @srini]

Interesting work

[clem 🤗 @ClementDelangue]

The HF science team just made async RL weight sync ~100x cheaper on bandwidth, and you don't need a shared cluster anymore.

The problem: every RL step, the trainer typically has to sync fresh weights to the inference engine. for a 7B in bf16 that's ~14GB. for a frontier 1T fp8 checkpoint, that's ~1TB; in bf16 it would be ~2TB. per sync.

The insight: between two RL steps, ~99% of bf16 weights are bit-identical. at RL learning rates, the optimizer is whispering and bf16 literally cannot hear most of it. the stored bf16 bits don't change.

What they shipped in TRL: only the changed elements get encoded as a sparse safetensors file, dropped into a Hugging Face Bucket, and fetched by vLLM. on Qwen3-0.6B, per-step payload goes from 1.2 GB to 20 to 35 MB. This is exactly what we built Buckets for: S3-like object storage on the Hub, Xet-backed (so even full snapshots only transfer the changed chunks).

The cherry on top: we ran a FULL disaggregated training where:
- the trainer lived on one box
- vLLM ran inside a Hugging Face Space
- the Wordle environment ran in another Space
- weights flowed through one Hub bucket

no shared cluster. no RDMA. no VPN. no NCCL across clouds. just HTTPS and a bucket.

one GPU + a Hugging Face account is now enough to do real disaggregated RL. multi-replica inference fleets across regions become a small devops exercise, not a research project.

Full write-up: https://t.co/CG115IjT0q

Open source RL keeps eating the moat!

[Srini Rajagopal @srini]

@citrini Wait till it comes to Microsoft and Amazon raising capital, everyone needs to rush out before the big ipos

[Srini Rajagopal @srini]

I would be surprised if othe hyperscalers AWS, Microsoft don’t do an equity raise similar to Google before the OpenAI and Anthropic fund raises to have enough dry powder

[Srini Rajagopal @srini]

@HannaHajishirzi Congratulations

[Dwarkesh Patel @dwarkesh_sp]

Recently met @srush_nlp and he started giving me an impromptu lecture on how targeted on-policy self-distillation works.

I asked him if I could record it on my iPhone.

The basic idea is this: if the model made a mistake at some point in the rollout (for example, calling a tool that doesn't exist), we want to discourage this specific error, but we don't want to just learn from the final reward, because it's a very noisy signal spread out over the whole trajectory.

So we have another model read this trajectory and figure where the error was made. It simply inserts some hint tokens to the part of the trajectory right above where the mistake was made.

Now with these injected hint tokens, have the model run a forward pass. You're not having to regenerate a new rollout - aka no new decode required.

The hint causes the model to assign lower probabilities to the error tokens. You then trains the original model to match these new probabilities, teaching it to downweight that specific mistake.

[Srini Rajagopal @srini]

@TheRohanVarma Congrats

[Srini Rajagopal @srini]

@nikunj There is also TPU, Google cloud

[Srini Rajagopal @srini]

@darshil Where is it happening

[Srini Rajagopal @srini]

@jukan05 @jukan05 how often do you change it? When will update it next?

[Srini Rajagopal @srini]

@inventur_es @joinshiftX Haha

[Srini Rajagopal @srini]

@mukund Interesting, time for you to move to to Bay Area :)

[Srini Rajagopal @srini]

@dsp_ Good stuff, stateless finally

[Srini Rajagopal @srini]

@bubbleboi @bubbleboi thoughts on TPU versus Trainium? Is there any writeup on this? thanks!

[Srini Rajagopal @srini]

@BrendanFoody Don’t fight the tide

[Srini Rajagopal @srini]

@KobeissiLetter @WisemanCap does this change your plan

[Srini Rajagopal @srini]

Excited for what the team is building!

[Jagannath Putrevu @jputrevu]

🚀 After a year of quiet building, I’m excited to officially announce @champ_hq out of stealth. We're also announcing our $8.5M Seed round led by @Redpoint with participation from @defyvc , @Max , @svangel, and a great group of angels. Watch the quick launch video below

[Srini Rajagopal @srini]

Crazy given all the progress that @sama is so bullish on the next 12 months

[Srini Rajagopal @srini]

Next year is going to be highest model improvement ever @sama

[Srini Rajagopal @srini]

Single agent for consumer and multiple agents for enterprise as per @gdb, Wdyt?