Rishab Bala

PhD student @VT_CS

[Rishab Bala @Sub_RBala]

🌟New paper alert: The Master Key Hypothesis🌟

Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.

Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.

[Rishab Bala @Sub_RBala]

@johnhewtt @GeorgeMorgulis Very interesting read!
We also find that steering vectors can be used to capture and transfer reasoning capabilities such as CoT, and the steered model can match/surpass prompting. Low-rank signal transfer seems to be very promising!
Paper: https://t.co/vQLzjv7axu

[Tu Vu @tuvllms]

I am in Rio de Janeiro, Brazil for #ICLR2026 @iclr_conf. Would love to connect and chat about any LLM research topics, explore collaboration, or discuss potential funding opportunities for my lab at @ Virginia Tech.

[Rishab Bala @Sub_RBala]

@thecekbote If the W_o and next FFN already do the mixing once per layer, isn't IHA functionally similar to MoE in the sense that you mix heads/residuals?

[Rishab Bala @Sub_RBala]

@universeinanegg Our recent work shows that reasoning can also be approximately linear in nature: https://t.co/vQLzjv7axu. But I do agree that the manual searching is required to find these maps/vectors

[Rishab Bala @Sub_RBala]

🌟New paper alert: The Master Key Hypothesis🌟

Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.

Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.

[Rishab Bala @Sub_RBala]

Work done in collaboration with @linusdd44804 @SharmaRituraj19 @anjiefang, Fardin Abdi, Viktor Rozgic, Zheng Du, @mohitban47, @tuvllms

Paper: https://t.co/vQLzjv7axu
GitHub: https://t.co/N2CPT13IHh

[Rishab Bala @Sub_RBala]

🌟New paper alert: The Master Key Hypothesis🌟

Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.

Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.

[Rishab Bala @Sub_RBala]

Based on our findings, we introduce the Master Key Hypothesis and postulate the convergence of capability representations across model families and scales.

[Rishab Bala @Sub_RBala]

The 3 self-distillation papers seem to be extremely similar in the method and only differ in how the feedback is generated/incorporated. They are also only compared to SFT (known to be the weakest method), while incorporating feedback is also done with other PO methods. Not quite sure of the takeaways, but the improvements and continual learning settings look good!

[Rishab Bala @Sub_RBala]

@giffmana @ArashVahdat @HaoZhao_AIRSUN What is the prompt for generating an image like this?

[The Sanghani Center at Virginia Tech @SanghaniCtrVT]

@therealthapa One more @SanghaniCtrVT paper at #EMNLP2025:

Efficient Model Development through Fine-tuning Transfer
Main proceedings

@linusdd44804 @Sub_RBala @tuvllms (all VT) w/@fyliufengyuan, @kandpal_nikhil

https://t.co/OGQa84ots4

[Tu Vu @tuvllms]

Excited to share that our paper on efficient model development has been accepted to #EMNLP2025 Main conference @emnlpmeeting. Congratulations to my students @linusdd44804 and @Sub_RBala on their first PhD paper! 🎉

[Rishab Bala @Sub_RBala]

@blelbach Will there be a recording?

[Rishab Bala @Sub_RBala]

@ryolu_ @cursor_ai Before all that can we get a way to remap autocompletes from tab to another button? And a way to get partial edits?