Katherine Tieu

CS PhD @UofIllinois @ideaisailuiuc | Research Scientist Intern @AIatMeta

[Katherine Tieu @kthrn_uiuc]

‼️We propose to learn temporal positional encodings for spatial-temporal graphs, L-STEP, in our #ICML2025 paper.

Even simple MLPs can achieve leading performance in various temporal link prediction settings!

📄 Paper: https://t.co/ioapcELCD5
💻 Code: https://t.co/sQWMlBZ1N6

[Chinmay @ChinmayKak]

insane good read on why sequence IS scales well with compute and token level(GRPO) and Geometric Mean based IS (GSPO) does not scale well, especially with async RL and why biased token IS works well at lower batch size.

[Yilun Xu @xuyilun2]

1/ It's been so much fun building Muse Image/Video from scratch with the team. The story of Muse Image's agentic behavior did not begin as a designed feature. It began as an surprise to us, from the self-refinement behavior to the test-time scaling!

https://t.co/cK8Yn4XE47

[Rosinality @rosinality]

https://t.co/6CwbbLgVHM

Benchmark of optimizers (and now we have a tier of optimizers).

[Sagnik Mukherjee @saagnikkk]

Bit late to the party. But will be there @icmlconf in Seoul 🇰🇷 !

Our paper in the intersection of RL and optimization will be presented as an oral presentation on 7th July in Hall C at 10:00 AM.

Please drop by !

[Lilian Weng @lilianweng]

A super long overdue (3+ years?) post on scaling laws.

Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.

The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.

https://t.co/HP26eJvjHB

[机器之心 JIQIZHIXIN @jiqizhixin]

Cool, Code as Agent Harness!

Researchers from UIUC, Meta, and Stanford introduce "Code as Agent Harness": a unified framework where code serves as the operational substrate for agent reasoning, acting, and execution-based verification — not just the output.

This approach outperforms fragmented agent designs by enabling executable, verifiable, and stateful multi-agent systems across coding assistants, GUI/OS automation, embodied agents, scientific discovery, and enterprise workflows.

Code as Agent Harness

Paper: https://t.co/s4Kx42o4ab
GitHub: https://t.co/aEJnIy5GML

Our report: https://t.co/3sqErhyZX1

📬 #PapersAccepted by Jiqizhixin

[Xiuying Wei @ICML🇰🇷 @XiuyingWei966]

The Renaissance of Sparse Attention (old dilated like Longformer/Longnet, compressed like DeepSeek, query-aware like MiniMax) vs. Hot linear attention/recurrence: Two separate lines of long-context scaling. We have a series of works with @caglarml that bridges them together! 🚀👇

1️⃣ RAT (NeurIPS '25): Interpolate between the recurrence of full-sequence compression and the attention of full-token access.
💡We use a light recurrence for short chunks, then apply softmax attention across them (as dilated attention). Note: This is highly similar to HCA in DeepSeek V4, but we call it as dilated attention, and our recurrence naturally has PE!

2️⃣ RAT+ (ICML '26): Flexible RAT with only one dense pre-training, which can then be flexibly switched at inference to different compression factors (opt. local windows) or hybrids.💥It flexibly and drastically saves KV cache with high accuracy! (vs. failures of GQA and StreamingLLM with 64x reduction on RULER/NIAH).
💡Our flexibility comes not from multi-config training (which ruins optimization), but from fundamental architectural redesign.

3️⃣ Recent finding: Beyond dilated patterns, dense RAT+ significantly boosts query-aware sparsity! (e.g., top-k block pattern in MiniMax).

🌟 Our Final Insights: Instead of just patching inference-time sparse attention post-hoc, we should also consider upstream architectures that are born to handle different levels of downstream sparsity more faithfully and flexibly!

[Nikhil Nayak @nikhilnayak268]

1/8 We published a new paper from @fastinoAI: Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers.

Main idea: adaptive optimizers like AdamW, Sophia, and Shampoo do not just have noisy updates. Their stochastic preconditioned updates are biased in specific, correctable ways.

Paper: https://t.co/zvIWHWyU2B.

[George Grigorev @iamgrigorev]

MSA paper is out

https://t.co/QQXr2vAt5p

[Xiuyu Li @sheriyuo]

Qwen Tongyi Lab proposes RLCSD, a simple but important critique of on-policy self-distillation.

Their key observation is that the distillation signal often concentrates on stylistic tokens rather than task critical reasoning tokens. As a result, the model may learn differences in phrasing, brevity, or tone instead of the reasoning process that actually determines correctness.

RLCSD addresses this with contrastive supervision that focuses learning on task relevant tokens. A nice example of a hidden failure mode in a technique that many people are rapidly adopting.

RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
Paper: https://t.co/irgUrl7nsm

[Penghui Qi @QPHutu]

We propose DRPO: a soft version of DPPO🔥

Since PPO, clipping/mask-based trust regions have long outperformed smooth divergence regularization like KL, even though the latter one feels more principled. 👺

We found two missing pieces:👇

1️⃣ Weight the regularizer by |advantage|
- Otherwise, the trust region geometry changes dynamically and optimization becomes unstable.

2️⃣ Use the right divergence
- What matters is not just “regularization”, but the trust-region geometry induced by the gradient. DPPO-style geometry works much better than PPO-style geometry in LLM.

These insights lead to DRPO, which delivers the most robust and best overall performance across algorithms, even outperforming original mask-based DPPO. 🚀

This project is an amazing collaboration with @ExplainMiracles, @NickZhou523786, Wee Sun Lee, Liefeng Bo, @TianyuPang1 . Do follow them if you are interested in this work!

📄 Paper: https://t.co/Un4QNoqUzZ

💻 Code: https://t.co/hPQp91sVUk

[Li Jiang @louieworth]

New blog post: On-Policy Distillation — Promise, Pitfalls, and Prospects.

OPD combines on-policy rollouts with dense teacher supervision.

But it is not a free lunch.

I discuss three failure modes and introduce our new paper.

https://t.co/xU35CqoMi3

[Patrick Jiang @patpcj]

Introducing Harness-1, a 20B search agent trained with a state-externalizing harness.

> frontier-level long-horizon search, rivaling Opus-4.6 and outperforming GPT-5.4

> Context-1-level cost and latency

> externalizes candidates, evidence, verification, and search history

> open-source

[Tilde @tilderesearch]

https://t.co/ShmmDeeNuP

[Nathan Lambert @natolambert]

Nvidia joined the multi-teacher, on-policy distillation (MODP) gang! Is industry standard post-training right now.

The multi-teacher SFT to RL that Microsoft did in their first model was the standard established by DeepSeek R1. I expect MAI 2 to be MODP.

[Yifan Zhang @yifanzhang_]

Introducing Self-Distilled Policy Gradient.

Token-level rewards, credit assignment, self-distillation.

RL and distillation are converging toward the same idea:

Policy gradients, it always has been, it always will be.

https://t.co/RJeRFUTeyz

[Xiuyu Li @sheriyuo]

On-Policy Distillation breaks badly when teacher and student distributions diverge: teacher supervision on student-generated tokens yields unreliable gradients and can outright fail optimization.

TrOPD fixes it with three clean choices.
1. trust-region on-policy learning runs OPD only where the teacher provides reliable supervision, dodging the reverse-KL estimator's instability under distribution mismatch.
2. outlier estimation handles unreliable regions via gradient clipping, masking, and forward-KL fallback.
3. off-policy guidance lets the student continue generation from teacher prefixes using forward KL, pulling on-policy exploration toward reliable regions.

The reason to care: every post-training pipeline using distillation hits this exact failure mode silently, the gradients get noisy, training loss looks fine, downstream accuracy collapses.

Trust Region On-Policy Distillation
Paper: https://t.co/jKviMKYChr

[Hanna Hajishirzi @HannaHajishirzi]

MAI-Thinking-1 is out!

Excited to share what we are building and how climbing from scratch (no distillation) actually works: simple recipes, rigorous science, self-distillation, patience, and great infra.

Check out our tech report has the full story of our RL climbs.
https://t.co/aLW40sWz4d

[elie @eliebakouch]

WOW microsoft new "MAI Thinking 1" model comes with a 109 page tech report that looks REALLY detailed, this is amazing

[Jiaxin Wen @jiaxinwen22]

New post: "Generalization Dynamics of LM Pre-training"

Most people (including me) assume that LMs smoothly mature from pattern-matching to generalizing.

This mental model is wrong. The true dynamics are stranger, and far more fascinating!

We call it Mode-Hopping.