Heyang Zhao

PhD @UCLA, || Previously undergrad at IIIS, @Tsinghua_Uni

[Dongruo Zhou @DongruoZ]

[1/7] Recent breakthroughs in LLMs’ mathematical ability are genuinely surprising.

I recently solved a problem I had been unable to solve for seven years: the optimal acceleration rate for first-order methods under high-order smoothness assumptions in nonconvex optimization.

[Qingyue Zhao @ZhaoQingyue]

🚀 Saturday afternoon Pavilion 4 P4-#4211! Our work takes a critical step towards a sharp understanding of offline policy learning under f-divergence regularization. (1/4)

[Kaixuan Ji @Kaixuan_Ji_19]

Come check out our poster on regularized bandits if you're interested in decision-making under KL (or other) regularization!

I won’t be at ICLR this year, but feel free to reach out to my wonderful collaborators @ZhaoQingyue and @HeyangZhao for a chat!

[Pan Xu @iampanxu]

#ICLR2026 We revisited stochastic linear bandits under heteroscedastic noise and find that the “total variance” isn’t the right complexity measure for variance-aware regret.

We show a bound that is strictly sharper than previous results using the harmonic mean of the variances.

[Qingyue Zhao @ZhaoQingyue]

Divergence-regularized RL objectives come back in the limelight under the name of alignment & reasoning, but the convergence of these algorithms against regularized objectives had not been completely characterized even for offline cases.

[Xuheng Li @xuhengli_]

Check out our new algorithm for contextual dueling bandits, FGTS.CDB!

🎲1st algorithm for contextual dueling bandits based on Thompson Sampling!

📈Theoretical guarantee & superior experiment behavior!

🔗https://t.co/hyPQBGgxMm

Joint work with @HeyangZhao @QuanquanGu [1/3]

[Kaixuan Ji @Kaixuan_Ji_19]

🔥Excited to share our recent research on query-efficient RLHF! Introducing Active Direct Preference Optimization (ADPO), a new approach that improves DPO performance on Open-LLM-Benchmark with just half the queries. Discover how ADPO eliminates the significant demand for querying preference labels.🚀[1/4]

Paper: https://t.co/VhwMM8VWVn
A joint work with @JiafanHe , and @QuanquanGu 👏

[QIWEI DI @di_qiwei]

🌟Excited to announce the acceptance of our paper on dueling bandits at #ICLR 2024! Dueling bandits play a pivotal role in reinforcement learning from human feedback (RLHF). Our research studies the impact of the labeler's confidence, specifically the variance of comparison probability in the Bradley-Terry-Luce model, on learning the reward model. We demonstrate how leveraging variance information accelerates the learning process, leading to lower regret. Looking forward to seeing you in Vienna!

Many thanks to my coauthors Tao Jin, @FrankYueWu1, @HeyangZhao, Farzad Farnoud, and @QuanquanGu

[Quanquan Gu @QuanquanGu]

Give someone a fish, and you feed them for a day; teach someone to fish, and you feed them for a lifetime.
Elevating from Weak to Strong with Self-Play Fine-Tuning (SPIN) for All LLMs. Empower, Evolve, SPIN!

https://t.co/6xjvCxkKj9

[Junkai Zhang @JunkaiZZ]

🌟 Excited to be at #NeurIPS2023 in New Orleans, Dec 10-17! Can't wait to meet with all my friends!

Catch our paper presentation in poster on Tue, Dec 12, from 10:45 AM to 12:45 PM CST. We'll also present our two workshop papers. Join us for some insightful discussions!

[Quanquan Gu @QuanquanGu]

📢 Excited to share our latest research on improving human-AI communication! 🤖💬 We introduce 'Rephrase and Respond' (RaR), a simple yet effective method that enhances LLMs’ understanding of human questions. Check out how RaR improves #GPT4 performance by resolving ambiguities & can be integrated with Chain-of-Thought (#CoT) for more robust AI responses.

🌟This work is led by @Yihe__Deng and an exceptional team of students @WeitongZhang @_zxchen_

Paper: https://t.co/NXcE4WNvSo
Project: https://t.co/QOzXpQ5j4n
Code: https://t.co/TQN1XYcW4X
HuggingFace: https://t.co/M3ClYX8Ang

Key Insights:
👉 Human input is key to LLM response quality. Crafting clear, detailed questions is crucial as our different thought frames may lead to AI misunderstandings.
👉 To tackle the disparity between human and LLM thought frames, we introduce RaR, which prompts the LLM to rearticulate the given question, and respond.
👉 Our experiments demonstrate that RaR significantly improves the performance of various GPT models across a wide range of tasks.
👉 We introduce formal mathematical formulations for both CoT and RaR and show that RaR is different and complementary to CoT. Through empirical analysis, we illustrate the importance of question quality—it should be prioritized before enhancing the model’s reasoning capabilities! 🧵1/N

[Quanquan Gu @QuanquanGu]

Very excited to be at the helm of the AI for Science initiative at ByteDance Research. Our unwavering commitment to reshaping the scientific discovery landscape using AI is a journey to watch.

Today, we release CryoStar, a state-of-the-art open-source tool for Cryo-EM heterogeneous reconstruction. It's the epitome of innovation in merging AI with the world of structural biology.

Project: https://t.co/xDQHqyaqE4
Code: https://t.co/xI6aPRsb3s
Paper: https://t.co/kUyuXi3R0w

Join us on this remarkable journey of exploration, from AI breakthroughs to groundbreaking discoveries in science. Stay connected, as more models and open-sourced tools are on the horizon!

[Quanquan Gu @QuanquanGu]

I believe the message in the paper is straightforward and uncontroversial. However, it seems there might be a misunderstanding in @abacaj 's interpretation. Pre-trained transformers can effectively acquire in-context knowledge for tasks related to their pre-training data and generalize to those tasks, but they cannot generalize to tasks significantly distinct from their pre-training contexts. In fact, the generalization ability of transformer-based in-context learning is discussed in terms of task diversity in https://t.co/kgHZc3dewd by @AllanRaventos @SuryaGanguli et al., and task complexity in https://t.co/eOX62RuUiY by @uuujingfeng et al.

[Yiwen Kou @YiwenKou666]

[1/6] 📢 Curious about the implicit bias in 𝐧𝐨𝐧-𝐬𝐦𝐨𝐨𝐭𝐡 neural networks trained with gradient descent (𝐆𝐃)? Find the answer in our latest #NeurIPS2023 paper! We study two-layer ReLU and leaky ReLU networks on nearly-orthogonal data.
Link: https://t.co/VlJPUQlvV6

[Jiafan He @JiafanHe]

RL with linear function approximation is a well-studied topic, but achieving the optimal regret guarantee in linear MDPs remains an open problem. Excitingly, our work proposes an innovative algorithm (LSVI-UCB++) that provides a solution to this challenge! 🚀 Check it out! (1/2)

[Heyang Zhao @HeyangZhao]

Paper link: https://t.co/5UyfeTyGIX
Coauthored with @DongruoZ @JiafanHe @QuanquanGu. (3/3)

[Heyang Zhao @HeyangZhao]

Excited to share our work "Optimal Stochastic Online Linear Regression" at #ICML2023 !

What is the optimal regret bound of online linear regression with stochastic noise?

Find us in Poster Session 3 @ Exhibit Hall 1 #328 from Wed 26 Jul 11 a.m. HST to 12:30 p.m. HST. (1/3)

[Heyang Zhao @HeyangZhao]

TL;DR: We study the problem of online linear regression in the stochastic setting. We provide a sharp analysis of the classical follow-the-regularized-leader (FTRL) algorithm and also prove a lower bound , which indicates that our upper bound is nearly optimal. (2/3)