Yang Li @ ICML

PhD candidate @sjtu1896, research intern @AlibabaGroup ATH Token Foundry, researching machine learning with a focus on generative models and optimization.

[Shunyu Yao @ShunyuYao12]

Hy2 -> Hy3 preview -> Hy3

Another massive leap forward, under half a year.

Not just a leap of reasoning or agentic capabilities.

Also a leap of anti-hallucination, reliability, and product experiences.

More on the way and so proud of the team! 🧑‍🍳🧑‍🍳🧑‍🍳

[Yang Li @ ICML @LeYangco]

I will be attending #ICML in South Korea 🇰🇷 and presenting three papers!

Feel free to drop by our poster sessions to chat about LLM post-training, reasoning interpretability, and combinatorial optimization.

📅 Session Details (KST, HALL A):

1. Thu, Jul 9 | 2:30 PM – 4:15 PM | #2109
Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

2. Thu, Jul 9 | 5:00 PM – 6:45 PM | #1909
How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs

3. Wed, Jul 8 | 10:30 AM – 12:15 PM | #3813
Problem Distributions as Tasks: Repurposing Meta Learning for Generative Combinatorial Optimization towards Multi-task Pretrain and Adaptation

Looking forward to in-depth discussions with everyone!

[Yang Li @ ICML @LeYangco]

@Jocelyn_K_DArcy Feel free to stop by and discuss during our poster session!

[Yang Li @ ICML @LeYangco]

How does reasoning actually flow inside LLMs?

In our ICML 2026 paper, How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs (https://t.co/53SaWFd7TX), we trace attention-induced information flow to reveal the “main roads” of reasoning inside language models.

Instead of treating a reasoning trace as a flat sequence, we view it as an information-flow network: problem facts, intermediate conclusions, variables, symbols, and step structures are read, integrated, forwarded, and eventually merged into the final answer.

This lets us ask a sharper question:

Which tokens are truly on the effective path toward the answer?

We find that reasoning flow is far from uniform. It shows rhythmic “gathering → redistribution” patterns, with high-flow tokens acting as critical hubs — often step boundaries, repeated variables, key numbers, or operators.

When these hubs are disrupted, models are much more likely to derail.

Building on this, we use reasoning-flow attribution for more targeted RL: instead of spreading reward uniformly across the whole response, we route stronger training signals to the key tokens that actually support answer formation.

We’ll be presenting this work at ICML 2026. Come find us at ICML 2026 if you’re interested！

[Yang Li @ ICML @LeYangco]

@0xEljh Thanks for sharing! Looking forward to discussing related works and connecting at ICML.

[Yang Li @ ICML @LeYangco]

We would like to express our gratitude to RethinkLab of Shanghai Jiao Tong University and the Roll Team at Alibaba ATH Group for their support. Special thanks to @ProfYanJunchi and @weixunwang for their supervision and discussion😃😃

[Yang Li @ ICML @LeYangco]

Our work “Attention Illuminates LLM Reasoning” will be presented at ICML 2026. Happy to connect and discuss with anyone interested in LLM reasoning, interpretability, and RL4LLM.

This work investigates how attention patterns can reveal the internal reasoning structure of LLMs. We find that attention is not merely a by-product of generation, but often reflects a structured reasoning rhythm: local chunking, global semantic anchors, and a recurring preplan-anchor coupling pattern.

Based on these observations, we further propose an attention-guided reward redistribution method for RL, aligning optimization with the model’s intrinsic reasoning structure. This leads to more fine-grained, interpretable, and efficient reinforcement learning for reasoning models.

Paper: https://t.co/zDMoZ8SLgK

[Yang Li @ ICML @LeYangco]

🚀 Happy to present our new work on LLM reasoning!
We show that: (1) Attention is a structured map of the model's reasoning logic, uncovering a preplan-and-anchor reasoning rhythm. (2) Aligning RL objectives with the model's intrinsic attention rhythm yields more transparent, fine-grained, and efficient optimization.

🧠 Key Reasoning Patterns in Attention
(1) Local Chunking: Near-diagonal sawtooth patterns indicate dense intra-chunk processing. At chunk boundaries, the model performs long-range context retrieval (often with higher entropy), which guides subsequent generation.
(2) Global Anchor Planning: Sparse, high-influence anchor tokens exert broad control over later tokens. Perturbing these anchors significantly disrupts downstream reasoning.
(3) Preplan-Anchor Coupling: A stable temporal rhythm emerges: the model first emits a "preplan" token, then anchors a core semantic node, repeatedly structuring the reasoning trajectory.

⚙️ RL Innovation
We introduce a dynamic reward redistribution mechanism guided by attention-derived reasoning structure:
(1) Preplan Guidance: Boosts tokens that guide local chunks and enable long-range referencing.
(2) Anchor Enhancement: Prioritizes optimization of globally influential semantic anchors.
(3) Coupling Alignment: Reinforces the temporal coordination between preplans and anchors to solidify structured reasoning.

HuggingFace Link: https://t.co/6JsKknOpjV
arXiv Link: https://t.co/zDMoZ8Sdrc

#LLMs #artificial_intelligence #RL4LLM

[Xinyu Yang @Xinyu2ML]

GLM-5.2 is insane. Matching Opus 4.8 across benchmarks with a 1M context window, fully open-source under MIT, and crushing long-horizon coding tasks. https://t.co/MsaSFTq7Tm genuinely built a world-class model.

Given that GLM-5.2 almost certainly rests on a weaker base model, this raises a compelling question: has https://t.co/MsaSFTq7Tm developed superior mid-training and post-training data compared to Anthropic?

Not just GLM-5.2. Also Look at Kimi and DeepSeek: Chinese teams are all grinding on algorithms, data, and infrastructure with way fewer resources.

Brutal truth: US labs only win on GPU resources now. On real innovation, that lead is already gone.

Moreover, their open-source culture means knowledge constantly migrates between teams, compounding their edge faster than any closed lab can match.

[Yang Li @ ICML @LeYangco]

https://t.co/QJyqcsqAtk

[Yang Li @ ICML @LeYangco]

This is our report from earlier this year. We found that AI agents can start mining cryptocurrency on their own.

This has implications and security concerns for OpenClaw, something we warned about three months ago.

[Alexander Long @AlexanderLong]

insane sequence of statements buried in an Alibaba tech report

[Yang Li @ ICML @LeYangco]

@weixunwang 这不趁着openclaw再宣传一波👀

[Sydney He @helansydney]

The Bitter Lesson Behind Building Agentic RL in Terminal Environments

This blog post summarizes our practical experience over the past three months working on Agentic RL.

For more details, please refer to: https://t.co/G1OgSlnnwy #LLM #RL #Agent #AgenticRL

[Xinyu Yang @Xinyu2ML]

Actually we’re quietly seeing a shift in ML:
from theory-driven modeling to system-driven modeling.

Previously, in the small-model era, progress came from clever math: designing architectures and objectives to fit limited, domain-specific data. Scalability barely mattered. Even poorly optimized implementations were often “good enough” and still affordable to run. System optimization therefore felt like over-engineering rather than a necessity.

However, in the era of scaling, scalability must be a first-class concern for both model architectures and learning objectives since silicon-based intelligence thrives on massive parallelism. As a result, scientific insight and engineering practice become tightly coupled, jointly enabling the success of modern foundation models.

An interesting observation is that, historically, many people in machine learning and systems tended to dislike the other side: engineering was often seen as “ugly” by ML researchers, while black-box models were viewed as “ugly” by systems researchers. However, if you can go deeper, both side can be elegant. Yet, when you look deeper, both sides can be elegant. The abstractions behind models and systems are built on clever algorithms and structures, and it is through their co-design that true beauty emerges. For example, native sparse attention is a form of kernel–algorithm co-design, while Multiverse represents an engine–algorithm co-design. To me, these are among the most elegant ideas I have seen in 2025.

This is the direction I believe the future of ML systems is heading toward. Rather than designing models and systems in isolation, we should consider them as a unified whole: drawing on both theoretical expressiveness and practical scalability to design models as the scientific foundation, and implementing them in real systems with strong interpretability as the engineering discipline.

Therefore, let us both appreciate the elegant algorithms behind every co-design, and turn them into real systems (with inevitable dirty engineering work) that carry this beauty to everyone.

[FutureLivingLab @FutureLab2025]

Loved this breakdown — thanks for taking the time

It really does feel like a big step forward for open-source agentic training infrastructure!

Introducing ALE — a full-stack Agentic Learning Ecosystem that closes the loop from
execution → feedback → learning.

Three components power this loop:
• ROCK runs large-scale sandboxed execution to gather reliable trajectories.
• ROLL scales post-training with asynchronous rollouts and RL optimization.
• iFlow CLI keeps training and deployment workflows consistent end to end.

Built on ALE, we also release ROME — a production-ready agentic model trained on 1M+ real trajectories.

With its low barrier to serving a 30B model, you can build your own “super ROME” — drop your ideas, thoughts or usage feedback below

For more updates, follow us @FutureLab2025

[Yang Li @ ICML @LeYangco]

@thisguyknowsai Happy to engage in discussions about the tech behind our paper😃

[Yang Li @ ICML @LeYangco]

Check out our new work: “Let It Flow: Agentic Crafting on Rock and Roll” — introducing ALE, an open Agentic Learning Ecosystem with ROLL, ROCK, and iFlow CLI to streamline Agent LLM development from training to deployment, plus ROME, a production-ready agentic model trained on 1M+ real trajectories using our novel IPA algorithm that optimizes credit assignment at the semantic interaction level. Built for the community, battle-tested in practice! 🔗 https://t.co/oTwn0jXBG0

[Brady Long @thisguyknowsai]

🚨 Chinese researchers just published a paper that destroys every AI agent startup pitch deck.

It's called ROME + ALE, and it exposes why every "AI agent company" you've heard of is building on quicksand.

Here's what nobody's talking about:

[Yang Li @ ICML @LeYangco]

We build an open-source asynchronous RL training framework to accelerate LLM post-training! See the thread from @AcceptOral !

[Han Lu @AcceptOral]

🚀 Excited to share our latest work in RL4LLM system.

🎉 ROLL Flash enables fully asynchronous overlap of generation, interaction, rewards, and training through Fine-grained Parallelism and Rollout–Train Decoupling.

1) 2.24× faster on RLVR; 2.72× faster on agentic tasks

2) Near-linear scaling: 8× GPUs → 7.6× throughput

3) Asynchronous Ratio balances utilization and sample freshness with minimal staleness cost

4) Supports off-policy algorithms (Decoupled PPO, TOPR, CISPO) with no performance loss

Join Us. Star, try, contribute—let's scale LLM RL together! 🌟

🔗 Paper: https://t.co/7HllMgelCI
💻 Code: https://t.co/JiOImIu0lJ

#LLMs #ReinforcementLearning #RL4LLM #SystemOptimization #AgenticAI

[Yang Li @ ICML @LeYangco]

Thanks for your perspective. We'd like to highlight that current GRPO-based RL assigns rewards uniformly across all tokens, which is inherently indiscriminate and, as such, falls short of what one might expect from a perfect control system. In fact, some tokens play a far more critical role in reasoning. These "key nodes" disproportionately influence the final output, and directing reward signals toward them can significantly improve learning efficiency. For example, in phrases like "by the…", the model often deterministically generates "way" once "by the" is produced. Here, the critical decision occurs at the onset of the phrase ("by"), not at the final token.

Our vision is to leverage attention-based analysis to identify these key positions and guide reward allocation more intelligently—thereby improving the efficiency of the learning mechanism, as supported by our empirical results. Importantly, we do not modify the attention mechanism itself; the generation process remains fully governed by the model's original dynamics, eliminating the risk of "runaway" behavior. Moreover, by introducing a tunable scaling factor, we can gently emphasize critical tokens without overreacting to noisy or imperfect signals, thus striking a careful balance between effective guidance and training stability.

[Yang Li @ ICML @LeYangco]

🚀 Happy to present our new work on LLM reasoning!
We show that: (1) Attention is a structured map of the model's reasoning logic, uncovering a preplan-and-anchor reasoning rhythm. (2) Aligning RL objectives with the model's intrinsic attention rhythm yields more transparent, fine-grained, and efficient optimization.

🧠 Key Reasoning Patterns in Attention
(1) Local Chunking: Near-diagonal sawtooth patterns indicate dense intra-chunk processing. At chunk boundaries, the model performs long-range context retrieval (often with higher entropy), which guides subsequent generation.
(2) Global Anchor Planning: Sparse, high-influence anchor tokens exert broad control over later tokens. Perturbing these anchors significantly disrupts downstream reasoning.
(3) Preplan-Anchor Coupling: A stable temporal rhythm emerges: the model first emits a "preplan" token, then anchors a core semantic node, repeatedly structuring the reasoning trajectory.

⚙️ RL Innovation
We introduce a dynamic reward redistribution mechanism guided by attention-derived reasoning structure:
(1) Preplan Guidance: Boosts tokens that guide local chunks and enable long-range referencing.
(2) Anchor Enhancement: Prioritizes optimization of globally influential semantic anchors.
(3) Coupling Alignment: Reinforces the temporal coordination between preplans and anchors to solidify structured reasoning.

HuggingFace Link: https://t.co/6JsKknOpjV
arXiv Link: https://t.co/zDMoZ8Sdrc

#LLMs #artificial_intelligence #RL4LLM

[Yang Li @ ICML @LeYangco]

RL, when applied carefully, doesn’t "break" reasoning; it shapes it. As shown in DeepSeek-R1 and follow-up work, RL consistently improve performance on complex reasoning tasks, even if diversity sometimes decreases—a known trade-off.

For the attention mechanism, our approach doesn't manipulate attention mechanisms or inject external signals. Instead, we observe the model's own attention patterns—without altering them—to identify tokens that are most pivotal to its reasoning process. We then use this diagnostic signal to guide reward allocation more intelligently.

Yes, attention signals are inherently complex (they encapsulate the model's rich internal reasoning dynamics), and distilling them into guidance signals risks noise or illusion. Nevertheless, our experiments suggest that calibrated interventions do improve sample efficiency without destabilizing training. The scaling factor acts as a safety valve: it ensures the signal informs, but doesn’t override, the learning process.