Mingyi_Hong

Professor at University of Minnesota; Amazon Scholar

[Zhaoran Wang @zhaoran_wang]

The takeaway: we deploy LLMs inside loops — revise, search, evolve — and then train them as if a single forward pass were the whole job.

REVES flips that: make the training objective the deployment objective. Do it once, and the gain shows up across the entire family of revision-style harnesses. [9/9]

📄 https://t.co/AKscAWGfB6
💻 https://t.co/PN6FUEohQR

[Mingyi_Hong @MingyiHong01]

@jonp_07 is giving an oral talk on our HiPER framework today at the LM4Plan Workshop @ ICML 2026!
🕝 Time: 14:40
📷 Room: Main Program — Grand Ballroom 101–102

HiPER develops hierarchical RL algorithms that enable planning for long-horizon agentic tasks through explicit hierarchical credit assignment. If you're still around the conference, come check out the talk and our poster! https://t.co/cnzyNNdyN8

More details about the work https://t.co/2HIpaRckLb

[Jiangweizhi Peng @jonp_07]

1/ LLM agents still struggle with long-horizon tasks.
Why?
Because today's RL algorithms treat agent behavior as a flat sequence of actions, while humans solve problems hierarchically: we plan, pursue subgoals, and adapt when plans change.
Our new work, HiPER (accepted at ICML 2026 🎉), brings this hierarchical structure directly into RL training.

[WinsonChen @winsonchen108]

Excited to be presenting our work on Stitch CUDA at #ICML2026! 🎉

If you’re attending, I’d love to chat about LLMs for GPU programming, code generation, and software systems. Come stop by our poster!

Poster Session 8 #115
07/09 5:00 - 6:45PM

[Mingyi_Hong @MingyiHong01]

It is interesting to observe that middle-layers contribute much more significantly than other layers during RL training, and sometimes a single layer is sufficient to match the full RL performance. These results from @ZijianZhangNLP @yauchungyiu, Athanasios Glentis, Dawei Li and Rizhen Hu and collaborator @hongzhou__lin would open many new opportunities in RL👏🏼.

📝Paper: https://t.co/H80Vy5Qe5c

[ZijianZhang@UMN @ZijianZhangNLP]

[1/N]
🔍 Introduce our new paper: "Is One Layer Enough? Training a Single Transformer Layer Can Match Full-Parameter RL Training"
🤔 RL post-training (GRPO, Dr. GRPO, GiGPO...) updates all transformer layers uniformly — but do all layers actually contribute equally? We ran the most systematic layer-wise study of RL training to date and found something surprising:
🤯 Training a single transformer layer can match — and sometimes surpass — full-parameter RL training!
We validated this across Qwen3 (1.7B, 4B, 8B), Qwen2.5 (Math-1.5B, 1.5B-Instruct, 3B-Instruct, DeepSeek-Distilled-7B), spanning math reasoning, code generation, and agentic decision-making. The pattern is remarkably consistent: the best single layer reaches or exceeds full-parameter RL in every model we tested, and the high-contribution layers always concentrate in the middle of the network.
👉 Paper: Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

[Mingyi_Hong @MingyiHong01]

[5] Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding
Time: Thu, Jul 9, 2026 • 3:00 AM – 4:45 AM CDT
Location: HALL A #1904

Previous benchmarks evaluate unlearning with greedy decoding. However, LLMs are typically deployed with probabilistic decoding. If a model appears safe under greedy decoding, can we also claim it is safe under probabilistic decoding?

The answer, for virtually all existing unlearning methods? No! We show that by using our metric, Leak@k, the probability that sensitive information resurfaces within k generations. To address the high-leakage issue, we propose a novel algorithm, RULE, and evaluate it on TOFU and MUSE. On TOFU, RULE achieves zero leakage across a large number of sampled generations. On MUSE, it outperforms existing unlearning methods across most sampling budgets 𝘬.

🔥 Paper: https://t.co/wMHiCd9xhj
🔥 Code: https://t.co/IgfS70wgcw

[Mingyi_Hong @MingyiHong01]

I will be at ICML and together with my students and collaborators we will present the following works. I will be at the conference on the opening day.

If you're attending, please stop by our posters, and we'd love to discuss optimization, reinforcement learning, LLMs, and AI systems!

[Mingyi_Hong @MingyiHong01]

[4] StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning

Time: Thu, Jul 9, 2026 • 3:00 AM – 4:45 AM
Location: CDTHALL A #115
‼️ LLMs are getting good at writing single CUDA kernels — but real ML workloads aren't single kernels. End-to-end GPU programs depend on fusion boundaries, launch configs, CPU↔GPU sync, and data movement.

🚀 We introduce StitchCUDA, a multi-agent framework integrated with rubric-based agentic RL for end-to-end GPU program generation. On KernelBench it achieves 90% success rate and 1.5× avg speedup over PyTorch eager — 1.72× over multi-agent baselines and 2.73× over RL model baselines — with a 32B Coder that beats GPT-5.2!

🔥 Paper: StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
🔥 Code: https://t.co/jmJIwSdSVM

[Mingyi_Hong @MingyiHong01]

[3] GUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
Time: Tue, Jul 7, 2026 • 2:00 PM – 3:45 PM (Seoul Time)
Location: Hall A #1209
‼️ MLLMs power today's GUI agents, but they still can't reliably point. On dense, high resolution professional screens, mapping "click the Send button" to exact pixels breaks down, and recent 7B grounding models stall around 50% accuracy on ScreenSpot Pro.
🚀 We introduce GUI-Spotlight, a think with image grounding model that iteratively narrows its focus like a spotlight. It invokes crop, extract, and color tools to home in on the target, trained with a stabilized multi tool RL pipeline built on GSPO. On ScreenSpot Pro it achieves 52.8% accuracy using only 18.5K training samples, beating V2P 7B (50.6%, 9.6M samples) and GTA 1 7B (50.1%, 1.56M samples) with up to 500× less data, while staying competitive with 72B scale models!
🔥Paper: https://t.co/DTRFblXxnp
🔥Code: https://t.co/WCrq3U5tXI

[Mingyi_Hong @MingyiHong01]

[2] Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
Time: Tue, Jul 7, 2026 • 2:00 PM – 3:45 PM (Seoul Time)
Location: Hall A #2109

LLM pretraining still relies on memory-heavy optimizers such as Adam. While memory-efficient implementations exist, it remains unclear which components are most essential beyond SGD. This motivates our central question: "Can we design a memory efficient optimizer with minimum modifications to plain SGD that achieves state‐of‐the‐art pretraining performance?"

By following a bottom-up approach from vanilla SGD, we identify two memory- and compute- efficient components that can significantly improve performance: column‐wise gradient normalization and first‐order momentum only on the output layer. This leads to our simple optimizer, SCALE, which has SGD‐level memory while achieving Adam‐level performance.

🔥Paper: https://t.co/DTRFblXxnp
🔥Code: https://t.co/84XsWZmF0i

[Mingyi_Hong @MingyiHong01]

[1] HiPER: Hierarchical Plan-Execute RL for Multi-turn LLM Agents
Time: Tue, Jul 7, 2026 • 10:30 AM - 12:15 PM (Seoul Time)
Location: Hall A #111
LLM agents struggle with long-horizon tasks for one big reason: credit assignment. When the only reward comes at the very end, how does the model know which of its thousands of tokens actually mattered?
Our answer: stop training agents as flat policies. 🧠

HiPER factorizes a single LLM into a high-level planner (proposes subgoals) and a low-level executor (gets them done) — then trains both with Hierarchical Advantage Estimation (HAE), a provably unbiased, lower-variance credit assignment scheme that gives each level its own learning signal.

With Qwen2.5-7B-Instruct, HiPER hits 97.4% on ALFWorld and 83.3% on WebShop — beating the best prior methods by 6.6% and 8.3% — while training roughly 2.8× more sample-efficiently compared to GRPO. The biggest gains show up exactly where it matters most: complex, multi-stage tasks. ⚡
🔥Paper: https://t.co/aaRHVxDwxi
🔥Code: https://t.co/gLTt8M8lKY

[Mingyi_Hong @MingyiHong01]

@AM921543266 No this is just an observation but does not necessarily accelerate the training, as computing the middle-layer gradient does not necessarily save compute for RL.

[Mingyi_Hong @MingyiHong01]

Thanks @askalphaxiv for sharing our new results on some interesting observations about RL training.

[alphaXiv @askalphaxiv]

"Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training"

RL post-training usually updates every Transformer layer, but the gains are not spread evenly through the model.

Especially with how most of the improvement comes from a few middle layers, sometimes training just one layer matches or beats full-parameter RL.

So this paper turns it into a simple recipe. By training or boosting the high-contribution middle layers, they show that you can outperform standard full RL with fewer changes.

[Mingyi_Hong @MingyiHong01]

Congrats to @Yuanxin02 and @zhaoran_wang @ZhouRuida Xinyan Zhao and collaborators on REVES! I enjoyed seeing this idea come together—optimizing the reasoning process itself, rather than a single-shot objective, is an interesting direction for post-training.

[Yuanxin Liu @Yuanxin02]

Think about how you actually solve a hard problem. You don't get it in one shot. You take an attempt, look at the feedback, and revise. Again and again.

That's how LLMs are deployed too: wrapped in a harness (revision, tree search, evolution) that loops.

So why do we still train them for a single shot? We built REVES to fix that. [1/8]

[xm @xianruimeng]

The semi industry is very interesting today, let's talk about design bandwidth.

Look at what's happening: hyperscalers are pouring ~$450B into AI infrastructure this year, and their custom silicon (TPU, Trainium, MTIA) is delivering 40–65% TCO advantages over general-purpose GPUs. Most recently, there's OpenAI. Serious AI company has now seen this math. The demand for custom chips has never been broader.

But here's the asymmetry nobody prices in: fab capacity is booked, yes — and design capacity is worse. and yes, Intel #Intel @Intel_Foundry and @TerafabCorp @elonmusk is trying to fix this! A competitive ASIC still takes hundreds of engineers making thousands of judgment calls over 18–24 months. The world's best design house runs ~600 engineers for $1.6B in revenue at 20% gross margins. Headcount scales linearly with projects. You cannot hire your way to 10x more chips.

So the industry keeps attacking cost with reusable IP, chiplets, better tools. All real. All incremental. None of them change who makes the decisions. The structural unlock is different: whether AI can own the design decisions, ie. the flow — floorplanning, timing closure, verification triage — with humans reviewing instead of doing. Track engineering hours per tape-out-ready block and drive it down every project.

When design stops scaling with headcount, custom silicon stops being a privilege of hyperscalers. That's the actual story of this cycle.

[Mingyi_Hong @MingyiHong01]

Excited to see that strong agentic RL system improves SOTA for cuda generation. Nice collaboration with system group @UMNApexlab and @ZijianZhangNLP

[UMNApex-Lab @UMNApexlab]

https://t.co/T1NfbRBHmF

[Yifei Zuo @YifeiZuoX]

According to prior attempts by Min Li and @Haoxiang__Wang, many recent optimizer records tuned for Attention do not transfer directly to PLX. I therefore isolated two clean modifications of particular interest:

1. Hyperball
2. EMA-Nesterov

Below is what PLX can achieve under parameter-matched settings.

A thread. (1/5)

[Mingyi_Hong @MingyiHong01]

I have read many of Dimitri's books, often through multiple editions, throughout my research career. One of my first optimization papers was inspired by an open question he posed in his book on coordinate descent. I learned an enormous amount from his writing.

[Mingyi_Hong @MingyiHong01]

Saddened by the passing of Dimitri Bertsekas. His foundational contributions to optimization, dynamic programming, control, and reinforcement learning shaped generations of researchers, including many of us. His books and ideas will continue to guide the field for years to come.

[Mingyi_Hong @MingyiHong01]

@Fanghui_SgrA @yuanhezhang6 @btreetaiji @jasondeanlee nice work

[Mingyi_Hong @MingyiHong01]

Thanks @jonp_07 for this exciting work! Hierarchical RL gives agents a planning harness for long-horizon tasks. Exciting to see HiPER outperform the carefully engineered SkillOpt + GPT-5.5 pipeline on ALFWorld. 🚀

[Jiangweizhi Peng @jonp_07]

1/ LLM agents still struggle with long-horizon tasks.
Why?
Because today's RL algorithms treat agent behavior as a flat sequence of actions, while humans solve problems hierarchically: we plan, pursue subgoals, and adapt when plans change.
Our new work, HiPER (accepted at ICML 2026 🎉), brings this hierarchical structure directly into RL training.