Yuxin Chen

Professor of statistics and data science at UPenn

[Yu Huang @yuhuang42]

🚀 Can RLVR models find their own frontier?

In our #ICML paper, we prove that mixed-difficulty RL can induce an implicit curriculum: easier tasks become learnable first, then pull harder tasks into reach.

(1/n)

[IMS @InstMathStat]

Yuting Wei Wins Peter Gavin Hall IMS Early Career Prize

https://t.co/HTCJo0E8uJ

[Yuxin Chen @chenyx04]

Two super talented student collaborators @yuhuang42 and @Zixin_Wen developed new theory for length-generalizable CoT reasoning!

[Yu Huang @yuhuang42]

Excited to share our recent work! We provide a mechanistic understanding of long CoT reasoning in state-tracking: when do transformers length-generalize strongly, when they stall, and how recursive self-training pushes the boundary. 🧵(1/8)

[Yu Huang @yuhuang42]

Excited to share our recent work! We provide a mechanistic understanding of long CoT reasoning in state-tracking: when do transformers length-generalize strongly, when they stall, and how recursive self-training pushes the boundary. 🧵(1/8)

[Tong Yang @TongYang_666]

🚨 🔥 Multi-step reasoning is key to solving complex problems — and Transformers with Chain-of-Thought can do it surprisingly well.

🤔 But how does CoT function as a learned scratchpad that lets even shallow Transformers run sequential algorithms that would otherwise require deeper architectures?

In our new work https://t.co/tSoFRsgO39, we prove that even a 1-layer multi-head Transformer trained via gradient descent can perform multi-step symbolic reasoning on trees.

What we find:
🔄 The model learns to “turn around”: first generates goal→root path, then reverses to root→goal path—all within a single autoregressive pass.
🧩 Two attention heads learn to specialize and collaborate: 🧭 one head follows the path; 🚦 one head tracks the phase and triggers the flip at the root.
@yuejiec @yuhuang42

[Simon Shaolei Du @SimonShaoleiDu]

Zihan Zhang (https://t.co/tnU3e3SUBN) is a stellar postdoc working with @jasondeanlee, @chenyx04 , and me. He's a rising star in ML theory and on the job market this year 🔥🔥🔥

Representative results:
Settling the sampling complexity of RL: https://t.co/WddogVAFjZ
Optimal Muti-Distribution Learning (solved a COLT 2023 open problem): https://t.co/YHun9ejWW5
Anytime Acceleration of Gradient Descent (solved a COLT 2024 open problem): https://t.co/tiJI48i4f7

[Jason Lee @jasondeanlee]

https://t.co/rDwGk2pM8Q Our postdoc zihan slays another COLT open problem! https://t.co/KOwIgY2k8g

[Yuxin Chen @chenyx04]

@AdelJavanmard Congrats!!

[Yuxin Chen @chenyx04]

Excited to share our recent work that proves sharp convergence theory for the probability flow ODE, a popular sampler in diffusion models. Encouragingly, our theory unveils linear dimension dependency.

Arxiv:  https://t.co/OHCHHPiU7a
Joint with: Gen Li, Yuting Wei, @yuejiec

[Simon Shaolei Du @SimonShaoleiDu]

How to learn the best shared model across multiple data distributions — a unified paradigm with applications in robustness, fairness, and calibration? Our COLT 2024 paper shows how to do it optimally using Hedge! https://t.co/YHun9ejWW5. Also resolved 3 COLT 2023 open problems: https://t.co/z2w1LEAHkE

[Simon Shaolei Du @SimonShaoleiDu]

Excited to share our new paper on optimal sample complexity/regret guarantee for RL!
New techniques: statistically decouple value function from transition estimations.
Happy to chat more at #ICML2023
arXiv: https://t.co/yTXDuQYhBf
with Zihan Zhang, Yuxin Chen, @jasondeanlee

[Yu Huang @yuhuang42]

Score-based diffusion models often suffer from low sampling speed. Can we design a training-free sampler that converges provably faster than the popular DDIM or DDPM? Our new work answers this in the affirmative: https://t.co/IQOutJjg7O

Joint with: Gen Li, Timofey Efimov, Yuting Wei, Yuejie Chi, and Yuxin Chen