Sky Thanawat

CS student @UTokyo_News

[Sky Thanawat @skydddoogg]

Coding agents are evaluated with unit tests:

more tests passed = better model.

But if tests or feedback are accessible, models may learn to game them.

We introduce CapCode to detect suspiciously high scores, and CapReward to discourage them during RL.

🧵1/10

[Sky Thanawat @skydddoogg]

We have two caps 🧢 today. CapCode and CapReward. ICML 2026 AgenticUQ workshop. Poster 2804!

[Sky Thanawat @skydddoogg]

Coding agents are evaluated with unit tests:

more tests passed = better model.

But if tests or feedback are accessible, models may learn to game them.

We introduce CapCode to detect suspiciously high scores, and CapReward to discourage them during RL.

🧵1/10

[Johannes Ackermann @johannesack]

We showed that different Black-Box Optimizers are more closely connected than we thought!

Interestingly, these connections result in different convergence characteristics which we can control and use.

Check out the paper or come visit our poster next week!

[Shinnosuke ONO / 小野新之介 ✈️ ICML2026 🇰🇷 @shinnosukeono]

We present our paper "Mitigating Reward Hacking via Adversarial Robustness" at EIML@ICML2026!

We conjecture that reward hacking is often caused by flipped advantage-sign estimations, and propose SignCert-PO, a new algorithm built on the theory of randomized smoothing! 🧵

[Soichiro Nishimori @nissymori1]

Our paper "Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying" is accepted at #ICML2026 🇰🇷

Proposed a novel exploration objective called ReMax, evaluating best of multiple trials under uncertainty.

The objective comes from the basic question,

Why do RL agents need to explore?

We argue it is because
♻️ Agents are allowed to retry (otherwise, the rational choice is the current best action).
📈 Return is uncertain (otherwise, no point in trying suboptimal actions.)

ReMax naturally captures these intuitions by modeling the distribution of returns and evaluating the maximum over multiple retries, thereby encouraging agents to select actions that are currently suboptimal but highly uncertain.

The diagram is inspired by the Vector Policy Optimization (VPO) paper.

🧵1/n

[Sky Thanawat @skydddoogg]

CapBench was the beginning of our Cap series! 🧢

We now have

- CapBench to make benchmark data contamination detectable
- CapCode to make cheating in coding tasks detectable
- CapReward to prevent cheating during RL training

[Takashi @tksiiml]

Excited to share that CapBencher was accepted to #ICML2026!☀️

We propose a way to publish benchmarks openly while giving them a built-in alarm for test-set overfitting such as contamination. We hope benchmark creators will consider using CapBencher for more reliable evaluation.

[Sky Thanawat @skydddoogg]

Paper: https://t.co/60LwA8MV84

Blog posts:
CapCode → https://t.co/UU9AFTTLvw
CapReward → https://t.co/O6si6JxlHB

Github repos:
CapCode → https://t.co/IonBsqXbxp
CapReward → https://t.co/91QNqkLKW4

10/10

[Sky Thanawat @skydddoogg]

Coding agents are evaluated with unit tests:

more tests passed = better model.

But if tests or feedback are accessible, models may learn to game them.

We introduce CapCode to detect suspiciously high scores, and CapReward to discourage them during RL.

🧵1/10

[Sky Thanawat @skydddoogg]

The whole idea in one conversation:

A: My model got 98% on this benchmark! 🚀
B: That’s cap.
A: Fake?
B: Yes. That is a CapCode benchmark, where non-cheating models should not go beyond 50%.
A: …oh no. My model might be gaming the tests.
B: Use CapReward to prove no cap.

9/10

[Sky Thanawat @skydddoogg]

@MungMingYP 🤣

[Sky Thanawat @skydddoogg]

@MungMingYP แล้วสรุปปป เมื่อไหร่อะ🫢