Zhepei Wei

Ph.D. Student @CS_UVA, Intern @MSFTResearch | Prev @AIatMeta, @AmazonScience. Research interest: ML/NLP/LLM.

[Zhepei Wei @weizhepei]

😢RLVR is powerful but expensive
🤯Imagine using <20% RLVR training while achieving 100% performance?

Sounds surprising? We show that minimal RLVR training is enough to know where training is going, and predict future ckpts at no training cost!

📃https://t.co/fGODWWIjR1
🧵[1/n]

[Zhepei Wei @weizhepei]

@yumeng0818 @CapitalOne Many thanks, Yu!😃

[Zhepei Wei @weizhepei]

🎉 Honored to receive the @CapitalOne PhD Fellowship!

Many thanks to my advisor @yumeng0818 and my collaborators for their guidance and support throughout my PhD journey at @CS_UVA @UVAEngineers! 💙🧡​​​​​​

Excited to continue building more capable, reliable, and efficient AI systems!

https://t.co/LJzPFsFiz5

[Zhepei Wei @weizhepei]

@TimXu222575 Thanks for sharing, Shuyao!😃

[Zhepei Wei @weizhepei]

@_ueaj @tianhongzxy @WeiLin__Chen @ChengsongH31219 @jiaxinhuang0229 @yumeng0818 @kalomaze Indeed a very interesting idea! Thanks for sharing!

[Zhepei Wei @weizhepei]

😢RLVR is powerful but expensive
🤯Imagine using <20% RLVR training while achieving 100% performance?

Sounds surprising? We show that minimal RLVR training is enough to know where training is going, and predict future ckpts at no training cost!

📃https://t.co/fGODWWIjR1
🧵[1/n]

[Zhepei Wei @weizhepei]

@tanghyyy @yuanzhi_zhu @tianhongzxy @WeiLin__Chen @ChengsongH31219 @jiaxinhuang0229 @yumeng0818 Thanks for bringing the great works to our attention! They have been cited and discussed in our paper: https://t.co/rGtOpQqB2Y

[Zhepei Wei @weizhepei]

@_ueaj @tianhongzxy @WeiLin__Chen @ChengsongH31219 @jiaxinhuang0229 @yumeng0818 @kalomaze wait, any context here? did i join the party too late? 😅

[Zhepei Wei @weizhepei]

The paper and accompanying artifacts are now released — including 500+ RLVR checkpoints for studying training dynamics and extrapolation! 🥳🥳

📚 Paper: https://t.co/olkSYHFAHb
📝 Blog: https://t.co/H9xWxD6dlZ
💻 Code: https://t.co/0ZF1WBlfAr
🤗 Checkpoints: https://t.co/Uj4OrbpoQl

[Zhepei Wei @weizhepei]

😢RLVR is powerful but expensive
🤯Imagine using <20% RLVR training while achieving 100% performance?

Sounds surprising? We show that minimal RLVR training is enough to know where training is going, and predict future ckpts at no training cost!

📃https://t.co/fGODWWIjR1
🧵[1/n]

[Jinyuan Li @JinYuan99121]

Can process reward models know when NOT to trust themselves? 🤔

We introduce BetaPRM: a distributional PRM that predicts both step-level success probability and the reliability of that prediction.

Instead of only asking “how good is this step?”, BetaPRM also asks:
“how confident am I?” 🔍

[Zhepei Wei @weizhepei]

@yuanzhi_zhu @tianhongzxy @WeiLin__Chen @ChengsongH31219 @jiaxinhuang0229 @yumeng0818 This is exactly one of our baselines, and we’ve also outlined the key differences in our paper—which will be released soon together with other artifacts. Stay tuned!😉

[Zhepei Wei @weizhepei]

Not yet. The current evaluation focuses on math tasks since the RLVR training domain is solely math. It would definitely be interesting to see whether the extrapolated checkpoints generalize better (or regress less) than fully RLVR-tuned models on other domains.

What non-math tasks would you suggest we evaluate on?😃

[Zhepei Wei @weizhepei]

Kudos to our amazing collaborators: Xinyu @tianhongzxy , Wei-Lin @WeiLin__Chen , Chengsong @ChengsongH31219 , Jiaxin @jiaxinhuang0229 , and Yu @yumeng0818 🎉🎉🥳

[Zhepei Wei @weizhepei]

📢 Takeaway: You only need minimal RLVR training to know where the model is heading.

Observe the early training dynamics, then go extrapolate future checkpoints at no training cost!

Blogpost👇
https://t.co/UWUJ1CB8tR

🧵[10/n]