Yifang Chen

MTS @MicrosoftAI postraining, Ph.D. @uwcse and undergrad @usc. Data curation, reinforcement learning, and active learning.

[Yifang Chen @cloudwaysX]

Wow there is new progress in horizon free RL after so long time waiting 😁

[Runlong Zhou @VectorZhou]

🚨 New paper: “Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence.”

We give the first polynomial-time algorithm for time-homogeneous tabular MDPs whose regret term is both:

• completely independent of the horizon H, and
• asymptotically optimal in S, A, and K.

Our regret bound is

Õ(√SAK + S⁸A³)

matching the contextual-bandit lower bound Ω(√SAK) in the leading term.
Technically, we introduce:
• an S-dimensional discretization of the monotone optimal value sequence;
• a new cutting bonus for horizon-free optimism;
• a total-deviation bound controlling clipped variance independently of H; and
• a horizon-truncation framework enabling reward-aware exploration.

📄 https://t.co/mogFTNGbFK

Greatly thankful to collaborators Zihan Zhang, Maryam Fazel, @SimonShaoleiDu

[Runlong Zhou @VectorZhou]

🚨 New paper: “Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence.”

We give the first polynomial-time algorithm for time-homogeneous tabular MDPs whose regret term is both:

• completely independent of the horizon H, and
• asymptotically optimal in S, A, and K.

Our regret bound is

Õ(√SAK + S⁸A³)

matching the contextual-bandit lower bound Ω(√SAK) in the leading term.
Technically, we introduce:
• an S-dimensional discretization of the monotone optimal value sequence;
• a new cutting bonus for horizon-free optimism;
• a total-deviation bound controlling clipped variance independently of H; and
• a horizon-truncation framework enabling reward-aware exploration.

📄 https://t.co/mogFTNGbFK

Greatly thankful to collaborators Zihan Zhang, Maryam Fazel, @SimonShaoleiDu

[Yang Liu @nlpyang]

Excited to introduce our MAI Code model at Microsoft Build. As shared in the session, this is a MoE (5B active / 137B total) initialized from an MAI pretrained model and trained for real user scenarios with product harnesses. I’m proud to have served as the research lead for this effort, and even prouder of what the team has achieved. It’s a beast for its size. Stay tuned — a larger model could come :)

[Weizhu Chen @WeizhuChen]

We just released MAI-Code-1-Flash, a 5B-active small model. The team just finished this from a pretrained model to VSC and Copilot CLI production within a few weeks, so proud of their achievements. More details are here https://t.co/jx85ssYAea and a bigger model is on the way.

[Yiping Wang @ypwang61]

We improve a 32-year lower bound in a challenging open problem, Ramsey numbers, through simply scaling autoresearch.

⭕ Proves R(3,17) >= 93. Previous 92 bound were obtained in 1994.
Google’s AlphaEvolve (2026) matched previous result but did not beat it.

All could be done with Claude Code / Codex + a CPU server.

Graphs and evolving history are available at https://t.co/2kCsk9Otur

[1/n]

[Jifan Zhang @jifan_zhang]

New research paper with Anthropic and Thinking Machines

AI companies use model specifications to define desirable behaviors during training. Are model specs clearly expressing what we want models to do? And do different frontier models have different personalities?

We generated thousands of scenarios to find out. 🧵

[Ruizhe Shi @smellycat_ZZZ]

Previous works study the sample complexity of DPO and emphasize the role of samplers in online DPO. What about its role in optimization convergence rates?

Check out our paper at #ICLR2025 on convergence rates of online DPO with various samplers!

ArXiv: https://t.co/IVwe0nGepQ.

[Avinandan Bose @avibose22]

🧠 Your LLM should model how you think, not reduce you to preassigned traits
📢 Introducing LoRe: a low-rank reward modeling framework for personalized RLHF
❌ Demographic grouping/handcrafted traits
✅ Infers implicit preferences
✅ Few-shot adaptation
📄 https://t.co/KzkEFPqkr9

[Runlong Zhou @VectorZhou]

🧠 Ever notice how LLMs struggle with familiar knowledge in unfamiliar formats? Our new paper "CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models" tackles this head-on!
🔍 Our findings:
- Created a qualitative pipeline demonstrating problem we call cross-mode knowledge retrieval that goes beyond traditional consistency issues: we study highly diverse word styles and spurious correlations between the mode (format) and the knowledge
- Formulated the problem as memorizing random token sequences quantitatively
- Demonstrated dataset rewriting requires the cross-mode knowledge to match the exact order of non-cross-mode knowledge
- Developed CASCADE: pretraining with cascading sequences dramatically outperforms dataset rewriting & scaling up training time!

This is a great collaboration with @YiZhangZZZ
Arxiv: https://t.co/Z7Ajuawa45

#AI #LLM #NLP

[Yifang Chen @cloudwaysX]

@shibainu_vodka 你还在nyc吗？我下周就过去！也是从new Jersey 通勤😑

[Weizhu Chen @WeizhuChen]

We released Phi-4-mini (3.8B base in LLM), a new SLM excelling in language, vision, and audio through a mixture-of-LoRA, uniting three modalities in one model. I am so impressed with its new audio capability. I hope you can play with it and share with us your feedback. We also trained a reasoning model, achieving 90.4 on Math-500.
Model: https://t.co/QJx65DtQPv
Paper: https://t.co/Jxktctw1pv
Blog: https://t.co/rzaA0x6e6I

[Yifang Chen @cloudwaysX]

@shibainu_vodka 我发现我坐椅子从来不靠背只坐半个，所以好像那种椅子都差别不大

[Yifang Chen @cloudwaysX]

@shibainu_vodka 一模一样

[Simon Shaolei Du @SimonShaoleiDu]

Introducing StoryEval: our new video generation benchmark! Can a model present short stories like 'How to put an elephant in a refrigerator'?
arXiv: https://t.co/cl7YWezwEb

[Yifang Chen @cloudwaysX]

@JoeyZhouUT @shibainu_vodka 受用

[Kyunghyun Cho @kchonyc]

feeling a bit under the weather this week … thus an increased level of activity on social media and blog: https://t.co/vZA3gs70kl

[Yifang Chen @cloudwaysX]

@shibainu_vodka 这个确实，但就是感觉长时间住的话日常食物有点油腻

[Jing-Jing Li @drjingjing2026]

1/3 Today, an anecdote shared by an invited speaker at #NeurIPS2024 left many Chinese scholars, myself included, feeling uncomfortable. As a community, I believe we should take a moment to reflect on why such remarks in public discourse can be offensive and harmful.

[Yifang Chen @cloudwaysX]

East Exhibit Hall A-C #3505
Dec.11 4:30-7:30pm
Come to check out!

[Ruizhe Shi @smellycat_ZZZ]

RLHF aligns LMs to a fixed weighted combinations of rewards, but cannot retrain a new model for each user's preferred weightings.
To achieve customization, prior work merges the parameters of single-objective models. Can we do better?
We show that merging logits is better!