Langlin Huang

LLM Reasoning, LLM acceleration.
2nd. Year Ph.D. at Washington University in St. Louis.
Lucky advised by Professor Jiaxin Huang

[Langlin Huang @shrangoh]

Can adding pure nonsense to a prompt make an LLM reason better? YES! 🤯

Introducing LoPE (Lorem Perturbation for Exploration). We found that prepending meaningless pseudo-Latin placeholder text (Lorem Ipsum) acts like magic—helping GRPO escape the "zero-advantage" trap on hard math questions!

Paper page📄:https://t.co/wngev6Zp4c

Broadens LLM exploration: increases resample success rate by ~3x📈
Strengthens LLM reasoning: improves average math reasoning by up to 13%📊

Here’s how it works👇

[Langlin Huang @shrangoh]

🙏 Acknowledgments

This work was done during my internship at @AmazonScience , with my amazing mentor @hliucola . I sincerely thank all the contributors for their hard work and valuable suggestions: @MononitoGoswami @RLixinyu , @jprithwish , @kanakaris_nikos , @Patrick_Blöbaum, @PurakJain !

[Langlin Huang @shrangoh]

🧵 Can your teacher go off-policy in On-Policy Distillation?
Yes! An unfamiliar student prefix makes the teacher less certain, and its token supervision less reliable.
��� We introduce SCOUT (Student-COnditioned Updates of the Teacher), which adapts the teacher to student prefixes.
(1/10)

[Langlin Huang @shrangoh]

💡 Takeaway

A strong model is not automatically a good teacher for every student trajectory. Sometimes, the teacher also needs to learn how to teach its students.

SCOUT turns that idea into sparse, student-conditioned teacher updates.

Paper: https://t.co/DD4xVWm45Z
Project: https://t.co/kVFJJIOnZ7
(9/10)

[Langlin Huang @shrangoh]

🤝 SCOUT is compatible with other OPD fixes

Other methods improve how teacher supervision is used. SCOUT improves the teacher itself.

The two work well together: adding SCOUT to On-Policy Trust Region (OPTR), a component of TrOPD, improves both math and code performance.
(8/10)

[Langlin Huang @shrangoh]

⏱️ Sparse updates are enough
The teacher need not update at every student step.

Updating every 10 student steps preserves most of the gains while requiring fewer teacher updates.

SCOUT lets the teacher adapt to its evolving student without constant updates.
(7/10)

[Langlin Huang @shrangoh]

🧪What drives the gains: extra teacher compute or student-conditioned updates?

“OPD + Teacher GRPO” applies the same outcome-reward updates, but starts from the original question rather than student prefixes.

Extra teacher training alone does not match SCOUT’s gains. Student-conditioned updates matter.
(6/10)

[Langlin Huang @shrangoh]

🎓 SCOUT results in both a better-performing student and a more on-policy teacher.

The trained teacher reasons better from student-generated prefixes:
• Higher continuation accuracy (Fig. 3).
• Lower entropy and a narrower entropy gap (Fig. 4).

=> The teacher becomes better at teaching this student.
(5/10)

[Langlin Huang @shrangoh]

📈 Across four settings spanning teacher scales, model families, and math and coding, SCOUT consistently outperforms the OPD baselines we tested.

We use 3 training seeds, repeated evaluations, and significance tests against standard OPD.
(4/10)

[Langlin Huang @shrangoh]

🧭 A simple idea: let the teacher learn, too.
SCOUT = Student-COnditioned Updates of the Teacher.

• Student: standard OPD.
• Teacher: GRPO on continuations of student prefixes, using verifiable rewards.

We gradually lengthen the prefixes, increasing the difficulty during training.
(3/10)

[Langlin Huang @shrangoh]

🔍 How off-policy is the teacher?

We ask the teacher to finish student-generated prefixes. As prefixes grow longer, continuation accuracy falls and teacher entropy rises.

The teacher becomes less accurate and more uncertain on the contexts where it must teach.
(2/10)

[Jinyuan Li @JinYuan99121]

Self-evolving reasoning models often improve for a few rounds—then collapse. Why?

We identify two reward-hacking failure modes:
1) Invalid questions exploit answer-based rewards: missing conditions or nonexistent solutions produce inconsistent Solver responses, making flawed questions appear challenging.
2) Repeated tasks evade diversity checks: LLMs can vary wording more easily than deepen the mathematical challenge, earning rewards for superficial variation.

Introducing R-Quest: question validity and task novelty feedback to address both, taking a step toward sustainable self-evolution.

✅ Equip the Solver to recognize and reject invalid questions
🔍 Detect task repetition beyond surface similarity, without external models or tools
📈 Sustain reasoning gains through continued self-evolution

In long-term self-evolution experiments running for up to 10 rounds, R-Quest achieves a 17.32-point lead over R-Zero in the final round. Across two model families, R-Quest improves mathematical reasoning, with gains that also transfer to code and general reasoning.

Title: Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
📄 Paper: https://t.co/FbrClJK7jm
💻 Code: https://t.co/QDMfQRSrGN
🌐 Project: https://t.co/z8SVuE35Zj

[Jiaxin Huang@COLM @jiaxinhuang0229]

My student Chengsong Huang @ChengsongH31219 is on the job market! 🌟

He has many exciting work on LLM/agents self-improving and post-training, including R-Zero (ICLR'26), G-Zero (NeurIPS'26 spotlight 🌟), EnvHarness (600 stars within 1 month released) and many more! He is incredibly productive, full of research ideas, and definitely a rising star!

Please talk to Chengsong if your team is interested in hiring researchers in LLM post-training, self-improving, agents, RL and other related areas. He will also attend NeurIPS at Atlanta in December.

[Jiaxin Huang@COLM @jiaxinhuang0229]

This year our group has 3 papers accepted at #ICML 2026 with topics related to reasoning, reward models, and RAG! My PhD student Jinyuan Li @JinYuan99121 will be attending and presenting his work on training data efficiency for PRMs. Feel free to stop by and chat with him on any topics!

Our posters:
1️⃣ Training Data Efficiency in Multimodal Process Reward Models
🗓️ Wed, July 8, 5:00–6:45 PM KST
2️⃣ Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation
🗓️ Thu, July 9, 10:30 AM–12:15 PM KST
3️⃣ Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing

[Jinyuan Li @JinYuan99121]

Can process reward models know when NOT to trust themselves? 🤔

We introduce BetaPRM: a distributional PRM that predicts both step-level success probability and the reliability of that prediction.

Instead of only asking “how good is this step?”, BetaPRM also asks:
“how confident am I?” 🔍

[Langlin Huang @shrangoh]

We release our codebase at https://t.co/2qVSnShoi7.
Grateful to my supportive and patient advisor @jiaxinhuang0229💖. Thanks to my amazing collaborators @ChengsongH31219, @JinYuan9912, @damonnodemon, @YuyiYang217542 🤝.

[Langlin Huang @shrangoh]

Can adding pure nonsense to a prompt make an LLM reason better? YES! 🤯

Introducing LoPE (Lorem Perturbation for Exploration). We found that prepending meaningless pseudo-Latin placeholder text (Lorem Ipsum) acts like magic—helping GRPO escape the "zero-advantage" trap on hard math questions!

Paper page📄:https://t.co/wngev6Zp4c

Broadens LLM exploration: increases resample success rate by ~3x📈
Strengthens LLM reasoning: improves average math reasoning by up to 13%📊

Here’s how it works👇

[Langlin Huang @shrangoh]

Summary and Takeaways📝:
✨Prompt space perturbation is a shockingly simple yet effective way to broaden LLM reasoning exploration.
✨Lorem Ipsum-like random sequence represents an ideal perturbation: enough to diversify exploration, while preserving response quality.