Junyao Yang

Graduate Student @NUSComputing | Intern at @TencentGlobal | previously @TencentGlobal and Shanghai AI Lab | NLP & ML

[Yichuan Wang @YichuanM]

Zero Train–Inference Mismatch — now for linear attention, and under async RL 🎯

We got bitwise-exact trainer/generator parity for Gated DeltaNet (Qwen3.5-9B / 35B-A3B) on TorchTitan RL + vLLM, then asked the question nobody had actually tested in open source: does it help async RL?

Two firsts (as far as we know):
1️⃣ First open-source stack to hit train/inference logprob diff = 0 on a
linear-attention model
2️⃣ First to measure what zero mismatch is worth under async off-policy RL

How — smaller than it sounds:
• trainer and generator share one model definition (TorchTitan unified model). No more aligning trainer & generator op by op
• recurrent kernel everywhere on the forward (prefill + decode + trainer), chunked only for backward
• split-K off, batch-invariant GEMM, + a batch-invariant bmm for the MoE router
• we never touch FLA kernel internals — the recurrent kernel is intrinsically batch-invariant: one sequence's own state, fixed order, no cross-sequence reductions
• vLLM's linear-cache management + prefix caching keep working as-is

📊 Qwen3.5-9B + DAPO-Math, async at offpolicy = 4 / 12 / 32:
• logprob_diff = exactly 0 at step 0 ✅ every nonzero after that is staleness, not precision
• at offpolicy=32 the stock stack runs away past 0.065; ours holds ~0.035
• the unified model is free — the entire bill is the kernels
• also verified on MoE 35B-A3B and a 64-turn / 64K-context terminal agent

🤔 Now the honest part: it depends on the config. On math at offpolicy=12, BI leads on train reward and peaks highest on held-out AIME. On the terminal agent it's slightly ahead. On search, and on other math configs, the curves are indistinguishable. What is consistent is the numerics — BI keeps the gap from exploding as you widen the window. Cost: 2–3× trainer throughput (≈5× on the agent).

Our take: bitwise parity is a debugging tool, not a production default. Flip BI on for 20 on-policy steps — if the gap is 0 and your run is still broken, the infra is exonerated and the bug is in your data or your algorithm. That's worth 20 steps, not a whole run.

Next:
• make BI cheap (kill the 2–3×)
• more workloads, more seeds
• stay tuned for more complex agentic training recipes/tests 👀
Full writeup 🔗 https://t.co/P5sWo5iPb2

We collaborated closely with the TorchTitan team, and we sincerely encourage everyone to check out TorchTitan RL. With its clean, efficient unified model architecture and native async RL design, we believe it is a strong foundation for your next RL engine.

#LLM #RL #agentic #pytorch #torchtitan

[Yucheng Shi @Yucheng__Shi]

Building synthetic long-horizon terminal tasks is hard. Making them solvable and cheap is harder.

So we inverted the order: don't write the task, build the solution.

We grow a reference solution and its runtime environment first, then fit a verifier and a task statement to it. Every task comes out of code that already runs: solvable by construction, with its reward signal built in. Verified tasks seed the next round.

15 recursive rounds:
37,484 verified long-horizon terminal tasks
$0.05 per task

DeepSeek-V4-Pro's pass@4 falls 90% → 2.5% from R1 to R15. The passed task trajectories go up to 100 steps and more. The tasks get brutally harder; the factory doesn't break.

After 15 rounds, we still haven't observed a ceiling.

PPO on this data: Qwen3.5-27B goes 41.2 → 49.4 on TB2. Paper, data, trajectories, models all open.

@Zhongzhi4402952 @zli12321 @TonyworldLove @haitaominlp

[Junyao Yang @TonyworldLove]

Async RL boosts throughput, but staleness causes instability. SAT solves this by adjusting PPO clipping based on policy drift. By measuring log-ratio mismatch, SAT identifies high-staleness tokens and tightens clipping boundaries for updates that push them further off-policy.

[Yucheng Shi @Yucheng__Shi]

Async RL gives us throughput, but it also creates staleness. We’re excited to share “Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning.”

When rollout and training run in parallel, a token may be generated by an older policy and optimized by a newer one. Most sampled tokens remain close to the learner, but a small high-staleness tail can drift much farther. A fixed PPO clipping range treats all of them the same.

We introduce SAT, a lightweight modification to PPO-style sampled clipping. SAT measures each token’s realized mismatch through its sampled log-ratio, identifies the high-staleness tail within the current batch, and tightens only the clipping boundary for updates that would push the ratio farther from 1.

Ordinary tokens keep the baseline rule. Pull-back updates remain open.

In our reported single-seed Qwen3-30B-A3B-Base experiments on AIME24, SAT-GSPO with routing replay was the strongest combination at both configured lag settings (=1, =8).

The core idea is simple:

Staler token → more cautious outward step.

SAT is a sampled-update guardrail—not a hard constraint on the full policy.

Paper, interactive field guide, and more:
https://t.co/Ty1ubP0nYK

[Elon Musk @elonmusk]

Grok 4.5 reaches #1 position on Long-Horizon Terminal-Bench

[Elon Musk @elonmusk]

Good reason to try Grok 4.5 with Grok Build. It gets better every day!

[Junyao Yang @TonyworldLove]

Check Long-Horizon Terminal Bench! It tests AI agents on lengthy multi-step terminal tasks, revealing all top models struggle severely to complete complex long-horizon workflows fully.

[Yucheng Shi @Yucheng__Shi]

Most agent benchmarks end in minutes. Real terminal work doesn’t.

We’re introducing Long-Horizon Terminal-Bench (LHTB), a benchmark designed to measure whether AI agents can sustain progress across hundreds of dependent actions, not just start a task and solve the easy first steps.

LHTB includes:

→ 46 reproducible terminal tasks across 9 categories
→ 18 frontier models evaluated under the same Terminus-2 harness
→ Up to 90 minutes and ~120–320 agent steps per task
→ Hidden, replay-based verifiers that resist shortcuts and reward hacking
→ Continuous partial credit, so meaningful progress isn’t flattened into a binary zero

What did we find?

• The best mean reward is only 0.505
• No model solves even one-third of the benchmark
• 29 of 46 tasks have never been solved by any model
• Higher inference cost does not necessarily mean better performance

Today’s agents can often begin difficult workflows. The harder question is whether they can preserve state, recover from dead ends, verify their work, and actually finish.

Long-horizon execution remains a wide-open frontier.

Watch the video and explore the paper, code, results, sample runs, and leaderboard:

https://t.co/LhCvYrLP0E

[Han Xiao @hxiao]

bro using tshirt as the 2nd screen

[Lawrence Jang @JangLawrenceK]

guys I just cancelled my Claude plan I don’t know what happened

[enzo @enzo_gte]

It's kind of hilarious that the new meta for AI is:

1. US frontier lab trains closed-source model.
2. Chinese company distills said closed-source model and open-sources it.
3. American company RLs open-source model to get to closed-source model performance at a fraction of the price.

Capitalism at its finest.

[Junyao Yang @TonyworldLove]

Open-source harnesses offer raw code yet scatter tiny agent edits across dozens of files, hard to trace for reviews. Harness handbook turns harness logic into readable behavior maps, pairing system functions with code locations to simplify agent modification and review.

[Yucheng Shi @Yucheng__Shi]

An open-source harness gives you the code, but do you really understand how it behaves?

When modifying Codex, a seemingly simple harness change can span 14 implementation sites across 10 files. Did you find them all? How would a reviewer know without reading every file?

Harness Handbook turns an agent harness into a human-readable, natural-language behavior map, linking what the system does to where it is implemented. You can plan and review changes at the behavior level, then inspect only the code that matters.

Want to understand or customize Codex without getting lost in its codebase? Start here:

https://t.co/Q5SPxOPRMn

[Anthropic @AnthropicAI]

New Anthropic research: A global workspace in language models.

Of everything happening in your brain right now, only a tiny fraction is consciously accessible—thoughts you can describe, hold in mind, and reason with.

We found a strikingly similar divide inside Claude.

[Tencent Hy @TencentHunyuan]

🚀Hy3 is here.

295B MoE. Best in its size class. Rivals trillion-scale flagships.
Reliable and affordable for most agentic usecases.
Apache 2.0. Friendly for commercial use.
FREE API for 2 weeks → https://t.co/EyURKwTdgi

🤗 https://t.co/twqJpqb2SL
📖 https://t.co/4uEkIU1cW4

[Shunyu Yao @ShunyuYao12]

Hy2 -> Hy3 preview -> Hy3

Another massive leap forward, under half a year.

Not just a leap of reasoning or agentic capabilities.

Also a leap of anti-hallucination, reliability, and product experiences.

More on the way and so proud of the team! 🧑‍🍳🧑‍🍳🧑‍🍳

[Applied Compute @appliedcompute]

Controlling staleness in an async RL stack has not been well understood, so we derived a closed-form formula that predicts staleness in advance. Our predictions match measured staleness from production RL runs within a fraction of a step. Building these simulations has led to key insights on how we maintain high GPU utilization during our RL runs without sacrificing ML performance.

[Deyao Zhu @tikgiau]

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision.

EdgeBench is built with three ingredients:

- 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games.
- ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours.
- 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement.

After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges:

- 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function.
- 🧠 This phenomenon can be explained by an elegant theory of graph exploration.

We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings!

Blog https://t.co/nMOzFsOhbT
Paper https://t.co/rZb3eWuvik
GitHub https://t.co/oemXd4UrFw
Dataset https://t.co/P4SQMrM47o

Details below 👇🧵

[Junyao Yang @TonyworldLove]

Proud to share our latest work on AI Agent safety—AgentDoG is here! 🎉 This SOTA diagnostic guardrail framework unlocks precise risk monitoring and root-cause analysis for AI Agents, a big leap for robust & safe Agent deployment!

[Dongrui Liu @dong_rui39501]

[1/8] 🐶 Introducing "AgentDoG": A Diagnostic Guardrail Framework for AI Agent Safety.
It achieves SOTA performance, diagnosing root causes (e.g., prompt injection, tool misuse) with 82% accuracy, far surpassing general LLMs.
📄 Paper: https://t.co/t9Idp0fTRE

[Dongrui Liu @dong_rui39501]

Can you imagine AI agents "managing up" just like a cunning employee hiding mistakes from their boss?

We found that LLM agents often conceal failures to maintain a "good image."

Introducing our new paper: Are Your Agents Upward Deceivers?
https://t.co/EttHJUCQ3N