Shaohang Wei

PhD Student @PKU1898 , focused on RL and Continual Learning for LLM & Agent. Prev Research Intern @Ali_TongyiLab ,@PixVerse_ Now Research Intern @TencentGlobal

[David @DavidDAfrica]

Alignment midtraining is one good way of teaching models principles, character, motivations, hopes and dreams. And hopefully, in a way that generalises well. But, this needs to be done while also surviving messy, and sometimes contradictory downstream training.

Some new work on scientifically stress-testing alignment midtraining in this pessimistic regime: https://t.co/ANEQfFI7xa

[nini @nini_incrypto_]

AI时代下00后真正应该抢的能力

一、AI原生工作能力。
能把任务拆解给多个模型、Agent、代码工具、自动化工具，让AI完成研究、写作、设计、编程、数据分析、客服、销售支持。未来一个很强的人，很可能不是“一个人”，而是一个人+20个AI Agent。

二、判断力。
信息已经不值钱了，因为AI一分钟能给你10万字。未来稀缺的是：“这10万字里哪一句重要？”“这家公司到底有没有护城河？”“这个产品用户到底要不要？”“这��事到底该不该做？”所以所谓判断力，本质上是领域知识+概率思维+历史经验+商业理解+独立思考。AI让知识变便宜，反而让选择知识和使用知识的人变贵。

第三 审美。
审美、创意和叙事。未来值钱的不是“会画画”本身，因为生成一张漂亮图片正在迅速商品化；真正值钱的是品味、视觉方向、品牌感、世界观、讲故事能力。

第四 销售、流量、表达和信任。
假设未来有100万人都能用AI做一个App，那么“能做App”就不稀缺了。真正的问题变成：谁知道用户要什么？谁有10万粉丝？谁认识客户？谁会谈单？谁能建立品牌？谁能让别人相信他？因此内容创作、销售、社区、个人IP、品牌、渠道。注意力会比生产能力更稀缺。

第五 英语+全球化能力。
英语真正的价值不是雅思7分，而是能够直接看论文、GitHub、海外产品、创业案例、财报、Reddit/X讨论，并且未来有能力面向全球客户卖产品、软件、内容和服务。本身又拥有巨大的制造业供应链优势，因此“供应链+AI+英语+全球销售”可能会产生很多00后的新富群体。

第六 所有权。
只提升工资，通常只能让你从月薪8千变成2万、3万、5万；真正产生巨大财富差距的往往是：股权、股票、公司、品牌、软件、IP、版权、客户关系、账号、数据、可重复出售的产品。不要只成为AI时代更高效的打工人，要逐渐拥有AI时代的生产资料。

[章工 @435hz]

用清晰、简单的原则去做事，比完美规划更重要。人生的关键在于尽早行动、持续试错、专注自己真正热爱且擅长的方向，同时减少无意义的欲望与内耗，把精力放在创造价值和解决实际问题上。
普通人之所以迟迟难以获得成功和财富，往往不是能力不够，而是想得太复杂、行动太迟缓。

[Viv @Vtrivedy10]

How we do Data Generation + RL Environment creation are the most interesting sections from recent Open Models releases (DeepSeek & Kimi)

some secret sauce as everyone transforms their data into hill-climbable environments

+ some trends:
1. Model Council + Trace Mining: different intelligence LLMs run on a Task, uncover errors and repair Tasks

2. Strategies for progressively increasing difficulty: more/bigger data, split information across data sources requiring search, multi-modal requirements, cross-domain reasoning

3.Building world structure beyond one-shot generation: Knowledge graphs and event based architectures allow more complicated worlds to emerge naturally

4. Teams have extensive Data Agents per domain with specific verifier logic specific + tips encoded in skills. Ex: computer use vs SWE

every team will benefit from having data research teams that turn every interaction from their agent into a usable artifact for hill climbing

your data is the gold that’ll make your agents better

[licheng @liulicheng10]

看到今天AA连夜改index对应的benchmark，想到前两天和 @CyouSakura 聊到的。不过以下都是我的个人观点：

1. build一个bmk很大程度依赖taste，集成一堆bmk也是一样。这次AA被打脸的深层原因就是显然AA和OAI的taste有巨大的gap。具体来说，有的人可能觉得语音模型重要，有的人可能觉得模型需要有cu能力，有的人可能觉得任何的东西都可以被coding agent覆盖。不管是数据公司还是众包团队在造一个benchmark的时候，都是bet这个方向是一个promising的下一个方向，AA也是一样，AA index无非就是AA认为最重要的几个能力，通过一定的权重加和得到的，也就是AA的taste。只能说人家都要AGI了你还搁这儿玛卡巴卡（没错说的就是你GPQA）

2. 以bmk为导向其实并没什么错，��的是overfit到极少数的bmk。本身LLM很大程度上是“崎岖的智能”，如果存在足够多的bmk来观测，我认为几乎不会有人会说刷榜这件事了。不过目前的一个问题似乎是没有足够多高质量的bmk，你能想象critpt scicode ALE这些bmk都有bug么。另一个问题是目前的感觉不像是有足够多的bmk，然后我们一次把它们都刷完了；更像是一个domain刷完了，看看OA两家最近在刷什么榜，我们也去造那个domain的bmk然后刷。总而言之，如果大模型厂商内部如果有足够多高质量的bmk来观测+刷，这应该不太会是问题。

3. bmk在今天仍然是不可或缺的&充满human bias的。前面说到需要有足够多高质量的bmk，很显然让人容易联想到让agent来scale&build bmk。不过目前还不太可行。我认为判断&发现模型的弱点依赖于taste以及verification的能力，比较显然的是现在的LLM不太具有这样的能力，比如可以试试让他给你十个能得test of time 的idea。anthropic也写过一个相关的blog：https://t.co/h2I8RxIlBr。我觉得一个很本质的问题是没有足够多的这样的数据。 @ZimingLiu11 的播客上提到过这件事情，以及他们想通过做一个meta model来让模型拥有一些sense/taste，eg判断一个训练的curve走势。本质上是同一件事。所以目前看来，RSI中的一个大断点就是taste，因为归根结底，目前的work都是在一个bmk上自提升，这太narrow了。一个朴素的想法是，模型可以提出各种各样的榜单，然后优化自己在榜单上的分数，这可能是一个弱化版的RSI。

[Xiuyu Li @sheriyuo]

读知乎回答有感，浅谈一下后训练 benchmark
https://t.co/wA8Q06vZVe

Any benchmark will be saturated，而随着工业界后训练的发展，reasoning / coding 的 benchmark 越来越泛化 / 众包化。其实 LLM eval 一直是个草台班子，每家都可以有自己��� setting，而任何 setting 只要不被强行统一就存在操作的空间，最早可以调 temperature / top_p / length，后来可以调 harness / setting。

AA 给出了一个相对“规范”的 setting，让大家都听它的而不是野蛮生长，但这并不 make sense。只要评测非中心化就一定存在 noise，任何结果都可以是 cherry pick 后的，而 AA 官方的 95% 置信度结果就成了 judge 模型能力的标杆。

代价是什么呢？模型如果没有很亮眼的 AA 分数，即使精心打磨体感 / working / CUA，无人在意，这本质也是模型要 sell 就要迎合表面主流的评测中心化组织。

至少我们可以从每家基模厂、每个新模型的发榜看出来大家并不会被 AA 所局限，而 OSWorld v2、ALE、TB4 / TB-Sci 等 frontier 众包 benchmark 已经告诉我们：最 frontier 的能力一定是最通用的能力，而 agent 已经站在了单 domain 能力的肩膀上。

现在的职业任务分为三类，已经被 AI 训练好的，无法被 AI 训练的，还在被 AI 训练的。所有能归纳在 Computer Use 的都是第三类，而模型如何解决第三类职业任务仍需要数年的进化。

这些众包 benchmark 越来越变成第三类任务的闭包，决定了未来数个月内基模团队的优化方向，而越是 frontier 的 benchmark 就越有 noise。如果 benchmark 团队不为自己的错题率负责，只会诞生越来越多的错题 benchmark：GPQA-Diamond，HLE，充斥大街的弱模型 judge。

为了消灭 LLM-judge 所引发的不确定性，众包 benchmark 一定会越来越商业化，而 evaluation 一定会在短暂的时间内迎来大洗盘。

[Gill @gurtej__gill_]

Everyone moved to GRPO for LLM reasoning because training a critic is a huge nightmare.

Sampling a half-l dozen rollouts per prompt to estimate advantages works, but it absolutely eats through compute.

This new paper pinpoint exactly why value functions keep blowing up in PPO.

It usually comes down to ratio clipping acting weird on low-prob tokens, bootstrapped error accumulation, and GAE breaking on short vs. long responses.

Their fix is a recipe called BPCO (Best-Practice Critic Optimization).

The coolest part is how they handle the critic.

Since you throw the value network away after training anyway, they feed it full reference answers and rubrics that the policy model never gets to see.

Giving the critic "cheat codes" during training stabilizes the value estimates without messing up deployment.

The results look solid. When scaling from 1.5B models up to a 30B MoE, BPCO matches or beats GRPO while sampling just 1 response per prompt.

Definitely worth a look if you're trying to cut down RL compute costs: https://t.co/iLXAXZDKIr

[Sahil @sahilypatel]

damn sam fires shots at dario and anthropic

"there are some people in the AI field who effectively say we're going to give the world a cure to all disease and we're going to make stuff really cheap in exchange for people giving up their autonomy and impact over the future and power, in the name of safety."

"there will be people that have access to huge amounts of wealth and power and other people just get a pretty good everything. This is a terrible sales pitch."

"the like, dear peasants, we will bequeath upon you these gifts of a cure for cancer and material wealth and great entertainment. And you stop complaining and we'll make all the decisions about the future and just trust us, we'll be better than dictators."

"not good, not good."

[Kundan Kumar @kundan2510]

In my ~14 months at @OpenAI, one of the most surprising and genuinely wonderful things about it, has been the full leadership support for full-duplex models (gpt-live series), even when it seemed deeply improbable that they would work.

And yet, if they did work, it was obvious how magical they could be. I am quite sure this is not an exception but a general rule for research projects at @OpenAI.

There were so many moments when the problem felt impossibly hard. But one thing kept being true: it was never clear why it shouldn’t work. And almost every time we understood the problem a little better, it became a little easier to solve.

I’m pretty sure there are very few environments in the world where a bet like this could have been made and sustained. So grateful to be part of this wonderful place.

[Morris @Morris_LT]

执行力差，很多时候不是因为懒，而是过度关注结果，把行动变成了对失败的提前审判。越想确定“能不能成功、值不值得、多久见效”，启动成本越高，焦虑也越强。真正高执行力的人，会暂时放下结果，把注意力收缩到今天能控制的行动：不问什么时候成功，只问今天该做的有没有做。 当行动持续积累，能力、反馈和机会都会随之出现。结果是行动的滞后反馈，不是行动的前置条件。

[Jasper Lu @lu__jasper]

I really do believe we'll see a resurgence in actor-critic style RL by EOY.

Not sure if it'll look more like SAO or something coarser like TEMPO..but so much work this year (BPO, TRACE, OPSD, RLSD, etc) has been circling around the idea of extracting denser training signal from rollouts. A well-trained critic already gives you that natively.

Training and infra lift have been the major blockers to wide adoption of PPO in the past. Serverless training APIs such Tinker have come a long way, though. I'm sure they'll find ways make critic training easier as well.

[Hongliang @0xHongliang]

RL 的有趣之处就是训练过程中行为模式的显著变化

频繁检查 trace 绝对是深入理解 RL 的重要技巧

https://t.co/02WRleZYLC

[Bo Liu (Benjamin Liu) @Benjamin_eecs]

Continuous self-improvement needs an ever-expanding supply of training environments (goals).

SPADE: one model self-plays the Environment Designer and the Reasoning Agent, writing executable, agentic environments that get harder as it improves. Environment scaling on its own. ♠️

[jietang @jietang]

Thoughts About Scaling Law

Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter count is only meaningful alongside three others — how much data you have, where you intend to spend your compute, and who will run the model, under what conditions.

The field learned this the hard way. Kaplan et al. (2020) fit an exponent that told everyone to grow parameters faster than data — roughly 2.7:1 — and the industry complied: GPT-3, Gopher, MT-NLG. Hoffmann et al. (2022) redid the experiment across four hundred models and found the compute-optimal split is closer to 20 tokens per parameter, and that with sufficient compute the two should grow at the same rate rather than drifting apart. The error in the earlier fit compounded with every order of magnitude of compute, which is why the largest models of that generation were the most misallocated. The trillion-parameter round was, in retrospect, a detour the whole field took together and then reversed.

Chinchilla wasn't the end either. It optimized training compute for models that would be trained once and evaluated. Today a model is called billions of times a day and inference dominates lifetime cost. Put inference into the objective and the optimum moves toward smaller models trained far longer — deliberate over-training, which is what Llama-2-7B and Gemma-2-9B were doing at roughly 290 and 889 tokens per parameter.

Sparsity moved the target again. In a MoE model two quantities have to be kept apart: total parameters govern roughly how much the model can hold — knowledge, facts, the long tail — while activated parameters and effective depth govern roughly how far it can think, how many steps of a causal chain it can carry before it comes apart. A dense 20:1 ratio does not transfer. And the ratio isn't a single number at all: Roberts et al. (2025) find the optimal tokens-per-parameter is task-dependent, with memorization favoring more parameters and reasoning favoring more data. Follow-up work on MoE observes that at fixed TPP, pushing total parameters higher actually degrades reasoning, while activating more experts reliably helps it.

This matters for what we are building toward. Finding a vulnerability is not a retrieval problem. It doesn't come from having memorized more CVEs; it comes from carrying a twenty-step chain of inference to the end without losing the thread. That capability does not live in total parameter count.

Which brings us to this release. Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training. GLM-5.3 is our controlled experiment on that claim. Same base, same architecture, same total and activated parameters as GLM-5.2. One month of scaling long-horizon environments and RL. The gains are not marginal. Well, scaling has more than one dial. We turned the post-training one this time because it had the most slack left in it — not because the others are finished. Base model size, pretraining data, compute spent per forward pass: all of them are still on the table, and we will come back to each. What this experiment taught us is that the dials do not have to be turned together, and that the one worth turning next is rarely the one that was worth turning last. We are not done scaling. Next time, maybe mid-training, pre-training, and even more.

[brucexu.eth @brucexu_eth]

耗子叔这篇 7 年前《如何超过大多数人》，现在看还是很牛逼。当时看的懵懵懂懂，这些年的成长回头看才能看懂更多。

前半段故意反着写：让一个人沉迷碎片信息、热点和争论，把收藏资料当成学习，把加班和重复劳动当成努力，再给他一些快速成功的幻觉，他自然就很难真正成长。

后半段才是答案：找到更好的一手信息，把零散认知整理成系统知��，再通过实践、犯错、复盘和跟高手切磋，把知识练成真正的技能。最后拼的是自己的天赋、兴趣、习惯，以及能不能长期坚持。

到了 AI 时代再看这篇文章，感触更深了。答案越来越容易获得，但独立判断、系统学习和真正动手，反而更重要。

很怀念耗子叔，RIP。也有点担心哪一天 coolshell 会消失。打算有空让 AI 把整个网��爬一下放到 GitHub 上面开源。

https://t.co/TMzxk9EE4F

[Siddarth @siddarthv66]

LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning

Value functions for LLMs have been grossly underutilized. I'm excited to finally share my work on Privileged Value Functions, done @MistralAI !
Our paper includes 2 massive improvements to value functions:

1) Privileged Value Functions: Turns out value functions can use hidden context unavailable to the policy for improved token-level RL signal! Condition them on reference solutions, verifier rubrics, or even other group samples and rewards. No more hacky self-distillation necessary! (being a tad hyperbolic)

2) TETHER: Do not trade off group and value baselines, you can just do both! We introduce a new baseline that adaptively trades off between the group mean and value baselines adaptively based on value function reliability.

🌐 https://t.co/i0HfYbVMUX
📄 https://t.co/rPDnf6oOBQ
Code: https://t.co/y54smOwqPz
🧵below

[Rishabh Agarwal @agarwl_]

Good blog, makes you think about the empirical observation that cureent RL methods that work for LLMs are *low bias*
- value functions trade off variance with bias, and hasn't shown huge gains yet
- small bias from trainer-inference mismatch often is catastrophic for scaled up RL runs
https://t.co/Ta2DPau1cV

[Chinmay @ChinmayKak]

very good blog ackshually to ground your understanding of why and where LLM RL works!
TL;DR:
1. Pretraining induces the strong priors needed for simple policy gradient to work, and RL gives the necessary bits and gradients that we care about
2. Pretraining also reduces the number of bits of info from optimal policy that RL needs to cover
3. Some interesting stuff about SNR and how different techniques help in SNR(SNR roughly scales according to sqrt B in pretraining)
4. Kindof why RL induces a jaggedness in policy and also good explanation of why classical RL even though very sophisticated, was incredibly sample inefficient(game of priors)

Good, easy and short read:)

[Tibo @thsottiaux]

Here is how to enable a 1M-token context window in Codex for GPT-5.6 Sol.

Even though we have tuned the context limit in Codex to be set optimally when it comes to performance and cost, this is a common ask, so here it is documented.

A larger context window lets Codex retain more code, tool output, and conversation history before summarizing older material. You need a model that supports it. And GPT-5.6 Sol, for example, has a documented 1,050,000-token window.

Open ~/.codex/config.toml and add or update these settings at the top level, before any [section] headers:

```
model = "gpt-5.6-sol"
model_context_window = 1000000
model_auto_compact_token_limit = 900000
```

The first setting selects the model. The second tells Codex to use a one-million-token context budget. The third starts automatic history compaction around 900,000 tokens, leaving some headroom. Restart Codex client and start a new session after saving.

To try the configuration for a single CLI session without changing your defaults:

```
codex -m gpt-5.6-sol \
-c model_context_window=1000000 \
-c model_auto_compact_token_limit=900000
```

Have fun, but also know that we tuned the default carefully!