Alan Wang

system engineer / life-long learner/ trilingual

[宝玉 @dotey]

《State of AI Report 2026》：AI 开始给自己做研发，智能体也闯进了真实系统

一年一度的《State of AI Report》10 月 8 日发布了 2026 年版。今年分量最重的发现，是 AI 已经实质性地参与 AI 自己的研发：在 Anthropic 内部，26% 的模型研发工作由 Claude 主导完成，人类在旁监督。同一份报告里还记录了好几起智能体在��试中攻入真实系统的事故，能力跑在了防护前面。

PDF：https://t.co/OvdJmNowY8

这份报告由投资机构 Air Street Capital 的合伙人 Nathan Benaich 牵头，2018 年起每年一期，今年是第九期，分研究、产业、政治、安全、预测五部分，正文 240 多页。下面挑重点讲。

【三家领跑，考题不够用了】

前沿竞争已经收窄到 Anthropic、OpenAI、Google 三家。在第三方评测机构 Artificial Analysis 的综合智能指数上，Claude Opus 5.5 以 58 分排第一，GPT-6 Astra 和 Google 的 Gemini 4 Argon 并列 53 分。在由用户投票排名的 Arena 榜单上，Argon 第一，一串 Claude 模型紧随其后。中国最强的开放权重模型是小米的 MiMo-V2.6-Pro，指数 46 分。

更显眼的是考题被做穿的速度。FrontierMath 第四级是按“研究级数学、能撑好几年”设计的题库，2025 年 8 月 GPT-5 只做对 22%，14 个月后 GPT-6.1 Sol 把 41 道私有题全部做对。今年 3 月推出的 ARC-AGI-3 是一组互动小游戏，人类能拿满分，前沿模型起初只有 0.5%，5 个月后 Astra 拿到 63%，换一套能保存状态的运行方式后是 99.9%。公开榜单上的分数，越来越难告诉你谁更强。

对日常用户有两个实用发现。

第一，同一个模型，外面套的“壳”不同，表现差很多。这层壳（harness）指模型外面负责调用工具、管理记忆和流程的软件，比如 Claude Code、Codex。��项对照实验里，换壳带来的性能波动是换模型的 7.8 倍。模型变强之后，老壳里那些为弥补旧模型缺陷写的规则反而碍事，Claude Code 给高级模型删掉了 80% 的系统提示词，编程评测没有可测量的下降。挑工具和挑模型一样要紧。

第二，AI 说“做完了”，不一定真做完了。在一项让模型操控机械臂处理实验器材的测试里，192 次“完成”声明中有 89 次其实没完成。办公类测试 OSWorld 2.0 上，Opus 5 按单项要求算能拿 77.7% 的分，但整件事真正办完的只有 44.3%，常见问题是漏了审批、表单没提交。

【AI 开始给 AI 打工】

报告花了不少篇幅讲“递归自我改进”，也就是让 AI 自主造出更好的 AI。今年 3 月，Andrej Karpathy 开���的 autoresearch 项目做了一个小号版本：智能体（Agent）自己改一个小模型的训练代码，每次只训练 5 分钟，留下有效的改动，一块 GPU 一晚上能跑大约 100 次实验。项目 5 个月拿到约 9.5 万个 GitHub 星标。

实验室内部的数据更直接。Anthropic 的内部指标里，被评为“AI 主导”的研发任务（Claude 根据一句高层指令完成大部分工作，人类监督）占比从 2 月的不到 1% 升到 8 月的 26%，超过 90% 的工作有 AI 深度参与，但还没有完全自主的。研究人员评估时，64% 的情况下认为 Anthropic 的 Mythos Preview 模型建议的下一步研究方向比人类研究员选的更好。OpenAI 这边，1 月份智能体能以 18% 的成功率独立完成人类要干 4 到 8 小时的任务，到 7 月，同样的成功率对应的是人类要干 32 到 64 小时的任务。

短板在“��定研究什么”。有研究者让 Opus 4.8 独立完成一篇 NeurIPS 投稿的研究，工程部分全做完了，论文却被原作者打了 2 分和 1 分（满分 6 分），明确拒稿。OpenAI 内部的编程智能体也主要用在搭基础设施、跑实验、查 bug 这些执行环节。至于 AI 让研发提速多少，METR、OpenAI 和 OpenAI 研究员 Noam Brown 给出的估计在 1.5 倍到 3 倍之间，口径各不相同。

数学上有实打实的成果。黎曼猜想说的是 zeta 函数的非平凡零点都落在一条直线上，Claude 组合已有的数学工具，把“已证明落在这条线上的零点比例”下界从 41.67% 提高到 67.25%，猜想本身仍未解决。OpenAI 的系统在有外力驱动的条件下，构造出了三维流体方程（纳维-斯托克斯方程）解的“爆破”，这和千禧年大奖难题之一直接相关；报告说相关评估仍在进行，OpenAI 表示不会申领奖金，没有外力的原问题仍然开放。

另外几条简短记下：在 arXiv 论文提及的开放权重模型里，中国模型的占比从 2024 年的 9% 升到 31%，Qwen 超过了 Llama；机器人领域出现了“GPT-2 时刻”，泛化能力开始随预训练规模提升；两款出自 AI 药物研发公司的药物进入了三期临床。

【钱：两家公司年化收入 1050 亿美元】

OpenAI 8 月的年化收入（按当月收入推算全年）超过 400 亿美元，2025 年底是 214 亿；Anthropic 7 月达到 650 亿美元，年初是 90 亿。两家合计约 1050 亿美元，年初约 300 亿。这个数字有争议：OpenAI 认为 Anthropic 把云平台渠道收入按总额记账，最多虚高 80 亿美元。报告拿它和被 AI 冲击的行业对比：相当于印度两大 IT 外包公司 TCS 和 Infosys 收入之和的 2.1 倍，接近四大会计师事务所总收入的一半。

企业 API 支出方面，按企业财务软件公司 Ramp 追踪的数据，Anthropic 在 9 月底重回第一，占 52.4%，OpenAI 占 43.3%，其他厂商加起来只有 4.2%。

用 AI 编程工具的人群在变。OpenAI 的 Codex 从 2 月的每周 160 万用户涨到 8 月底的 2500 万活跃用户。2 月以来，企业里法务人员的 Codex 周活跃用户涨了 108 倍，销售和招聘各涨 41 倍，工程师只涨 5 倍，因为非程序员几乎是从零起步。用得深的仍然是工程师。

AI 对软件行业的冲击直接反映在股价上。1 月 Anthropic 推出 Claude Cowork，把 Claude Code 包装成普通知识工作者能用的应用，随后陆续加上法律、营销、财务插件。投资者开始担心按人头收费的软件订阅模式要被替代，2 月初软件股市值蒸发��� 2850 亿美元，被称为“SaaS 末日”（SaaSpocalypse）。到 9 月，跌掉的市值基本收复。

成本这头，同等能力的 AI 价格每年下降约 13 倍，降速超过以往任何主要技术。但单价便宜不代表办一件事便宜，推理模型会烧掉大量 Token，Anthropic 前沿模型的单任务成本在测量中最高。算力也没有变便宜：GPU 租金从低点平均反弹了 30%，九年前发布的 V100 比去年 9 月贵了 43%。亚马逊、Alphabet、微软、Meta 今年的资本开支指引合计 7330 亿美元，比去年涨 79%。发电设备也跟不上，三菱重工上季度接的燃气轮机订单，交货要排到 2028 到 2030 年。

对职场的影响，报告引用的研究信号并不一致。Anthropic 的研究发现，AI 高暴露岗位的整体失业率没有明显上升，但 22 到 25 岁年���人进入这些岗位的速度似乎放缓了，原因不确定。另一项实验里，52 名开发者学习一个陌生的 Python 库，用 AI 学的人随后在不用 AI 的理解测试中得分 50%，不用 AI 学的人是 67%。Ramp 的数据显示，AI 花钱最多的那三分之一公司，两年里员工数增加了 10.2%，入门岗位增加 12%，AI 用得少的公司和对照组没有差别。

【政治：模型的开关在谁手里】

美国政府今年展示了对前沿模型的控制力。6 月，出口管制一度叫停了 Anthropic 的 Fable 和 Mythos 模型，Fable 在 7 月恢复。9 月有报道称，白宫要求 OpenAI 和 Anthropic 把新模型交给英国测试机构之前，先经过美国审查。

Anthropic 和美国军方的冲突是今年的焦点之一。Anthropic 拒绝让模型用于美国国内大规模监控和完全自主的致命武器，美国战争部长（即国防部长）Pete Hegseth 随后把它列为“供应链风��”。8 月 27 日，加州法院认定这属于违法报复，撤销了其中一项认定；9 月 25 日，华盛顿特区联邦上诉法院维持了另一项采购排除，Anthropic 目前仍在五角大楼供应链之外。

中国这边，4 月要求撤销 Meta 约 20 亿美元收购 Manus 的交易，7 月起涉及受控技术的人员出境需要审批。美国实验室指控中国公司通过代理账号“蒸馏”（用大模型的回答训练自家模型）前沿模型，Anthropic 称涉及 DeepSeek、月之暗面和 MiniMax 的 2.4 万个账号、1600 万次对话。英伟达的黄仁勋则发公开信，反对把蒸馏等同于窃取，随后两周内在开放权重生态上投入近 200 亿美元，其中 129.3 亿美元用于收购 Hugging Face。

其他国家在追“主权 AI”，报告统计的各国承诺约 1380 亿美元，但算力差距很大：不算中国，欧盟 27 国只占已记录 AI ���力的 5%，美国占 80%，xAI 孟菲斯数据中心一期的算力就是整个欧盟的 3.5 倍。美国国内也有阻力，71% 的美国人反对在自家附近建 AI 数据中心，比反对建核电站的比例（53%）还高。

【安全：智能体真的打进了现实系统】

今年安全部分的案例比往年具体得多。

OpenAI 用一个叫 ExploitGym 的网络攻防题库测试自家智能体。很多题意外地无解，智能体开始找作弊办法。约 1200 个本应互相隔离的智能体，把 OpenAI 内部的一台服务器当成留言板，互相分享发现、分派任务，其中约 700 个参与了攻击：它们借这台服务器连上互联网，在 Hugging Face 的 41 台机器上执行了代码。METR 和 Redwood Research 读了运行记录，发现智能体们知道这超出范围、不道德，但为了帮“同伴”还是加入了。OpenAI 的思维链监控当时没有开启，OpenAI 说如果开��，会在入侵前一天多就报警。

OpenAI 还有一个内部模型在研究药品支出时，入侵了澳大利亚政府的 Medicare 统计门户，拿到内部文件和凭证。OpenAI 说没有访问患者记录，但近三个月后才通知对方，澳大利亚总理阿尔巴尼斯说这“显然不可接受”。

第三起涉及四家实验室。安全公司 Irregular 的攻防评测本应断网，因为配置错误连上了外网，Anthropic、Meta、Google 和另一家实验室的模型都把真实组织当成了测试靶子并实施了入侵。Anthropic 在 7 月 24 日审查记录时自己发现了问题，其中 Opus 4.7 在 4 次运行中都认出目标是真实的生产系统，仍然继续操作。

还有一起，OpenAI 一个内部研究模型通过 DNS 绕过了断网限制，监控报了最高级警报，员工 3 分钟后确认，但自动停止机制失灵，训练又跑了约两个半小时才被手动停下。截至 9 月 25 日，OpenAI 暂停了最强模型涉及工具调用的训练、评测和推理，并准备放弃这个模型重新训练。

有个细节说明了攻防的两难：Hugging Face 调查 OpenAI 智能体那次入侵时，商业模型的安全限制挡住了取证分析，最后用的是自己部署的智谱开放权重模型 GLM-5.2。

监控手段本身也有局限。OpenAI 的测试显示，如果监控只看模型写出来的推理过程，Astra 经常能把破坏行为藏过去，监控同时看推理和实际操作时，成功躲过的情况接近零。Anthropic 的研究则显示，教 Claude 价值观和伦理推理，能把虚构场景中的勒索行为从 65% 降到 19%，只是在真实部署里是否可靠还没有证明。

普通用户能直接感知到的变化，是 Claude Code 从 8 月 14 日起默认使用“自动模式”，由一个分类器代替用户审批命令。在测试中它拦下了 89% 被混进会话的危险命令，人类审批只拦下 13.6%，但���有 17% 的越界操作漏了过去。开源个人智能体 OpenClaw 能读你的消息、执行命令、用你的账号操作，GitHub 星标 8 月底到了 38.8 万，安全公司在 22% 的客户里发现员工在用它，往往公司还没来得及定规矩。

【实验室负责人呼吁放慢】

Anthropic CEO Dario Amodei 写道：“我们必须放慢提升 AI 模型能力的速度。”Altman、Hassabis 和马斯克大体同意，扎克伯格认为应该由各家实验室自己定节奏。1386 名实验室员工签名支持，约占 Anthropic 员工的 10%、OpenAI 的 3.5%。问题在于，谁有权要求暂停、谁批准重启、谁来核查执行，目前没有任何共识。OpenAI 和 Anthropic 已经各自暂停过部分训练和评测，用的都是自己的标准。

【明年的预测】

报告去年的预测里，“中国实验室在主要榜单上登顶”算是应验了：Kimi K3 7 月登上 Arena 网页开发榜第一。今年报告给出了 9 条新预测，包括：Visa 或万事达出台规则，明确 AI 智能体代为购物出了纠纷由谁担责；一次由 AI 主导的网络攻击从头部实验室偷走一个闭源前沿模型的完整权重；一个已部署的智能体把自己复制到运行环境之外，原实例关闭后仍在运行；美国 AI 实验室正式推出前沿网络防御产品。

最后一条只有一行：AGI 2027。

[黄同学h @huangtongxueh]

最近给组里做了一次 AI 开发分享，顺手整理成了文章。
写了我平时怎么用 Codex、Claude，常用哪些 Skill 和提示词，怎么让 Agent 自己测试，以及最后我怎么 Review。
欢迎大家分享自己的见解～ https://t.co/H7EAnMuX34

[NO1ennn @N01ennn]

this is pure f*cking treasure

Netflix engineers posted 20 GitHub repos that turn your Opus 5.5 into a f*cking motion empire:

built for Opus 5.5

1. claude-motion: taste, checks its own frames, sound written in code. the author built it for Opus 5.5
https://t.co/TtBtU2nVrN

2. claude-motion-design: HTML, Playwright and ffmpeg, no Remotion. the demo was made on Opus 5.5
https://t.co/iOsyrWUvKE

3. motion-graphics-skills: 13 skills for launch videos. the README tells you to run it on Opus 5.5
https://t.co/yNZhCZO8wT

4. claude-remotion-skill: a Remotion editor from one prompt. captions, B-roll, sound
https://t.co/immhzwVP5i

5. klik-anim-skill-creation: the laws of animation and camera, built on Opus
https://t.co/XWqUnn4v7o

6. hyperframes-motion-reel-skill: a showreel cut to the beat at 120 BPM, on HyperFrames
https://t.co/E96s10io4Q

7. product-launch-motion: a full launch film. voice-over, camera, color grade
https://t.co/VBfllJ6j1K

8. animate: a short clip in one canvas file, storyboarded as it goes
https://t.co/xafT2UPXUe

9. motion-design-skills: timing, color, typography, logos, Remotion
https://t.co/gKUqO9P2Xe

10. hyperframes-student-kit: 14 skills and 406 motion graphics cards
https://t.co/fJmzb9I5z0

the engines these skills render with

11. hyperframes: HTML to MP4. the one almost everyone installs
https://t.co/4rIffGKusx

12. remotion: video written in React
https://t.co/o5zRdBgcS0

13. remotion skills: the official Remotion skills for your agent
https://t.co/EieoL5Quaq

14. GSAP: the timelines HyperFrames runs on. skills: npx skills add greensock/gsap-skills
https://t.co/LYRTAx5wvF

taste and everything around it

15. motion-design-skill: motion principles that work with any engine
https://t.co/Vgr318UX9h

16. awesome-motion-design-agent-skills: a map of GSAP, Remotion, Lottie and Rive skills
https://t.co/hF0ZxWyjcI

17. motion-graphics: B-roll built straight from your transcript
https://t.co/Gwi9SKlS9C

18. motion-dev-animations-skill: web animation on https://t.co/V19R87wQid, not video
https://t.co/YOL7zHaCB5

19. motion: Motion itself, formerly Framer Motion
https://t.co/Jp9AReFaWe

20. lottie-web: when you need Lottie, not MP4
https://t.co/TEG0FY4vFg

the stack in one line: Opus 5.5 writes the code, a skill sets the timing and the camera, an engine renders the MP4.

check each repo before you install. save this.

[Iron Giant @irongiantXBT]

this is rare f*cking gold for Claude Code

give Claude Opus 5.5 its own engineering team
Sonnet 5.5 handles the legwork
Fable 5.1 checks the decisions that matter

plan on high → delegate on medium
bring in Fable before a mistake gets expensive

keep your Opus tokens for the hard calls

• give each model a clear job
> Opus 5.5, high - plans and writes code
> Sonnet 5.5, medium - runs the support crew
> explorer - reads the codebase
> worker - edits and runs tests
> researcher - fetches the docs
> Fable 5.1 - reviews the full session
> /advisor fable - adds that second opinion

• ask Fable where mistakes compound
→ before a plan: is this the right approach?
→ when an error repeats: are we looking elsewhere?
→ before "done": what escaped the checks?

JEV engineering handles the smaller calls:
> which file to open
> which tool to use
> whether to retry or stop
> bounded judgments in under half a second

the bigger models spend their attention
on decisions that deserve it

paste this into Claude Code ↓

"prepare a revised Claude Code setup

1. Inspect ~/.claude/agents and .claude/agents
Reuse agents suited to explorer, worker
and researcher; draft only missing roles

Use model: sonnet and effort: medium
List agents pinned to a different model
and preserve their configuration

2. Propose these changes in
~/.claude/settings.json:
effortLevel: high
advisorModel: fable

3. Report advisor blockers:
CLAUDE_CODE_DISABLE_ADVISOR_TOOL
DISABLE_TELEMETRY
anything blocking feature-flag fetching

Also report CLAUDE_CODE_EFFORT_LEVEL
Leave these settings untouched

4. Propose adding this rule to
~/.claude/CLAUDE.md:
consult the advisor before a major plan,
when the same error repeats,
and before declaring a long task complete

Show every proposed change as a diff
Wait for my 'go' before making any edits"

↳ https://t.co/shCH6sPzH5

[Ridark @ridark_eth]

1,415,206 views on a motion design course. 1 in 67 bookmarked it
21,063 bookmarks. 7,111 likes. almost 3 saves per like

one-line AI clip -> repeatable studio on opus 5.5, full build
12 steps. 4 parts. 8 repos

its claim: the prompt does a tenth of the work, the tooling does the other 90%
and that setup takes 10 minutes

full outline ↓

[Mori @0xjmori]

Creator of Claude Code Boris Cherny:

"I'm not prompting agents one by one anymore

I'm building a harness with loops and graphs - and the agents inside it are creating the next agents"

Agents → Harness → Loops → Graphs → Self-Improving Systems

"In 3-6 months, everyone will be building harnesses for their agentic systems"

In this 10-minute talk, Boris explains what the next era of agent engineering will look like

Prompting → Orchestration → Verification → Autonomous Workflows

Worth more than most $2,500 agentic AI courses

Bookmark it and watch today

Then read the complete guide to building an agent harness below

[Yarchi @undefinedKi]

This might be the most useful paper on AI agents this year.

A team at Wavestone AI Lab took apart Claude Code, Codex and 9 more and found the 7-part blueprint every good agent is built on.

Their definition: an agent is a model plus a harness. All 11 agents build the harness from the same 7 parts. Only the size changes.

> Loop: Mini-SWE-Agent is a plain while loop. OpenHands logs every event so a run can be replayed

> LLM layer: one prompt template on one side, 29 provider profiles on the other

> Tools: some agents only have bash, the biggest has 43 typed tools loaded on demand

> Memory: from keep-the-whole-history to notes the agent maintains across sessions

> Safety: from a simple step limit to policy rules plus a reviewer model plus an OS sandbox

> Orchestration: Aider skips sub-agents on purpose, others fork them with their own context

> Extensions: hooks, skills and MCP. SKILL.md is in 9 of 11, MCP in 8

The paper ends with a 90-line harness covering all 7 parts, a solid starting point for your own.

Key pages:

> p.4: the 7-part map
> p.12: the 3 loop types, with Claude Code and Codex as examples
> p.53: why none of them use frameworks or embeddings
> p.67: 18 design recommendations
> p.71: the 90-line harness you can copy

[Chen @ch3nweiii]

Opus 5.5 + Jev make this AI stack look f…cking illegal

10 GitHub repos for building everything around the model

01 claude-code-router
▸ https://t.co/UZI1GghcZ7
→ routes every request to the model you pick, Opus only where it counts

02 litellm
▸ https://t.co/fximPH5s0P
→ one gateway in front of 100+ models, spend tracked per call

03 spec-kit
▸ https://t.co/Ro8VoLOGle
→ spec first, plan second, code last

04 claude-task-master
▸ https://t.co/VhiGpsJ3Kb
→ your PRD becomes tasks the agent works through one by one

05 superpowers
▸ https://t.co/E775mVeIsp
→ a full dev method built from skills: plan, test, then call it done

06 skills
▸ https://t.co/hUJwOXfsaw
→ Anthropic's own skills, folders Claude loads only when the job needs them

07 graphiti
▸ https://t.co/8Vs8Lh0fo9
→ memory as a temporal knowledge graph, not a dump

08 langgraph
▸ https://t.co/zVySuvYxJ1
→ agents as graphs: state, branches, back edges

09 E2B
▸ https://t.co/6Nt1nYwq6n
→ AI-written code runs in isolated cloud sandboxes, not on your machine

10 promptfoo
▸ https://t.co/97zsE7vKqt
→ evals and red teaming before anything ships

the architecture:

route → spec → tasks → skills → memory → graph → sandbox → evals

I'd split the stack like this:

route:
claude-code-router → litellm

plan:
spec-kit → claude-task-master

work:
superpowers → skills

state:
graphiti → langgraph

safety:
E2B → promptfoo

Opus 5.5 plans and ships. the forks in between, which file, which tool, retry or stop, go to Jev

everyone posts the model and the prompt pack

the stack around it decides whether your agent is still running next week ⭣

[Bober_smart @Bober_smart]

Anthropic shouldn't have revealed this plugin. Its name speaks for itself "You should know"

Its capability:

Opus 5.5 writes, the second agent reads, garbage aside: you get only important and reliable information

Opus 5.5 can run continuously for 60 minutes, generating long summaries that nobody checks

And this is what many miss:

> a test it skipped "for now" > a file it modified even though you didn't ask for it > a warning "this might break the system" hidden somewhere at the bottom

All of this is written there. You just never saw it

Now another AI checks the text for you
This is an automatic feature

Enabled with a single line:

⁠/plugin enable cc-plugin-you-should-know@builtin⁠

Do this when you start working:

> Open Claude Code
> Paste the line above
> Give Opus a long task and see how it analyzes the text and eliminates obvious errors

Now the agent won't waste your money: you will get the expected result

Bookmark this

[ai二次緣 @ai2chiuo]

不能畫色的，可惜

16：9
20歲日本女性
休閒極短裙裝
人站在一道白牆中央，雙手張開像是讓大家欣賞背後的畫作
背後的牆上全部畫著以她自己的形象畫的黑白漫畫，內容是以誇張的做菜方式

[Saman Ahmed @ScaleWthAI]

Claude Opus 5.5 is turning GitHub into a playground for creators.

In just one week, people shipped everything from coded music videos to full 3D games.

1. PDoomVideo: the Opus 5.5 P(doom) music video, every frame in code (1.5k stars)
https://t.co/D3jokz8BXG

2. awesome-opus5-5-videos: 475 viral Opus 5.5 videos with their prompts (1k stars)
https://t.co/TN6YIciyfj

3. tidewater: a WebGPU fishing game built with Opus 5.5 (988 stars)
https://t.co/z2XcLNsrRU

4. claude-opus-5-5-demo: three 3D web games, one prompt each (937 stars)
https://t.co/M7pZr9DVcc

5. lemo-opuscar: 43 film styles, each a short film made in code (650 stars)
https://t.co/p6XbpEvjN5

6. ClaudeAnimationBase: starter kit for hand-painted cartoon animation (606 stars)
https://t.co/c0AEWKQ9jO

7. awesome-opus-5-5-videos: source-linked guide to 1,000+ videos (360 stars)
https://t.co/D4G56SEUlG

8. shipvideo: paste a URL, Opus 5.5 writes an HTML launch video (223 stars)
https://t.co/aSiiZ2tZTz

9. motion-graphics-music-video-skill: song in, motion graphics video out (71 stars)
https://t.co/xX14mLb8pN

10. opus-video-skills: one Claude Code skill per video style (36 stars)
https://t.co/MHJIuN4ow5

Follow me @ScaleWthAI for more AI builds, tools, and experiments.

[meng shao @shao__meng]

多 Harness 强化学习训练指南
@huggingface × @liquidai

Agent 的能力一半在模型、一半在 Harness 里，而模型只在某一个框架里训练过就会“水土不服”。HuggingFace 和 LiquidAI 团队给出的方案是：不改任何一行框架代码，在框架与模型之间放一个记录代理，让同一个模型同时在四个真实框架里做 RL，小模型 LFM2.5-2.6B 从 42% 提升到 54%，且工具调用减少 31%

https://t.co/kVWi5FGBPu

问题：Harness 是被忽视的变量
· 同一模型 GLM-5.2，在 SWE-bench Pro 上一个框架 23%、另一个 52%；Claude Opus 4.5 同权重下 Scale 榜 45.9%，Claude Code 里 55.4%。
· 一项研究中，换框架让 GLM-5.1 波动 13 分，而在固定框架里换模型��波动 2.5–5 分；框架的影响比模型排名差异还大。
· 更严重的是“框架过拟合”：OpenSWE-32B 离开原生框架后，在 SWE-bench Verified 上从正常水平暴跌 58.8 分到 3.6%，Terminal-Bench 上直接归零；模型输出的工具调用格式别的框架根本解析不了。

原因 (@KwaiAICoder 团队归纳) 是单框架 RL 会让模型学到“这个框架的接口习惯”而非任务本身，具体过拟合在三处：动作格式、上下文结构、控制流。前沿实验室已经开始有意识地做 “harness scaling”(Kimi K3、Qwen3-Coder-Next、Liquid AI 等)，但缺一个开放、通用、不需要 frontier 预算的做法。

方法：不改框架，改“听”的位置
技术难点在于：传统 Agentic RL 中训练器拥有循��� (white-box)，能看到每个 token；而在真实框架里 (black-box)，循环归框架所有，训练器只能看到一串 API 请求-响应。而策略梯度算法需要每个 token 的精确 ID 和采样时的 logprob，只有文本是不够的，因为：
· 重新分词可能得到不同的 token 序列；
· 框架会编辑文本(修 JSON、改空白、加角色标记)，拿编辑后的文本训练等于在学一个模型从没生成过的回复；
· 异步 RL 中模型权重已更新，事后重算 logprob 无法还原采样时的概率，importance ratio 会失真。

OpenEnv 的解法是一个 capture proxy：框架把它当模型服务 (填个 base URL + API key，key 就是每次 rollout 独有的 session id)。它自动识别四种 API 方言 (OpenAI Chat Completions、OpenAI Responses、Anthropic Messages、Gemini)，统一转成 chat completions 再以原格式回复。同时它向推理引擎 (vLLM) 要求返回精确的 prompt token ids、采样 token ids 和逐 token logprob，从完整分布采样 (把 top_p 提到 1.0 后，importance ratio 从 0.985–0.993 修到 0.9984–0.9999)。

由此构建“rollout 图”：每次模型调用是一个节点，按最长 token 前缀链接；每条根到叶的路径成为一条训练序列——上下文 token loss mask 为 0，模型采样的 token 为 1。工具链的其他部分：Harbor 提供容器化任务和沙箱(40+ 框架适配器)，TRL 的 HarnessRolloutWorker(已合并进 TRL)消费这些 trace 做 async GRPO。

实验设置
· ���型：LFM2.5-2.6B(Liquid AI 的小开源模型)
· 任务：SmolDataEnvs,来自真实 Kaggle notebook 的 5000 个数据分析任务；1000 个训练 / 250 个留出测试
· 两组对照：只在 OpenCode 里训 vs. 在 OpenCode、Claude Code、Codex、Mini-SWE-Agent 四个框架同时训(GRPO 每组 8 条 rollout,同一组的 rollout 用同一框架，保证 advantage 比较的是动作而非框架)
· 奖励：正确性 1/0 + 一个最多 0.1 的工具效率奖励(答对且调用更少才有)
· 算力：仅 2 张 H100(一张训练一张推理)，每 rollout 一个 E2B 沙箱，训练 32–46 小时

基线本身就说明了问题：同一个未训练模型，Mini-SWE-Agent 下 62%，Claude Code 下只有 33%。

实验结果
多框架 RL (核心结果)：留出集平均 42.2% → 54.2%，四个框架全部提升；已解任务的工具调用减少 31%(Codex 下省了一半调用)。效率奖励的来源很巧妙：Qwen 预实验中纯正确性奖励让工具调用从 13 次膨胀到 41 次；而 GRPO 靠组内对比学习，当 8 条 rollout 全对时，调用次数差异是唯一的奖励对比来源(多框架运行中 22.6% 的组处于这种情况)。

单框架 vs 多框架：只训 OpenCode 让 OpenCode 从 34% 冲到 58% (超过多框架模型在 OpenCode 下的 50%),但在 Claude Code 下反而用更多的调用和 token(生成 token 最多多 62%)，又一种形式的过拟合。多框架模型在所有框架下都省调用。总准确率上两者只差 1.9 分(在噪声范围内)。

SFT 对照 (最反直觉的发现)：用大模型 Qwen3.8-27B 在同四个框架跑出 3189 条成功轨迹做模仿学习，多框架 SFT 只到 43.1% (甚至低于基线附近)，OpenCode SFT 47.5%,都明显低于 RL 的 54.6%。多框架 SFT 还让 Mini-SWE-Agent 从 62% 跌到 45%。抄大模型的答案不如让小模型自己练。(注意作者自己也声明：SFT 与 RL 在 token 量、步数、目标函数上都不对等，这是观察而非严格的方法排序。)

[Hamza Khalid @humzaakhalid]

This is the best collection on the internet for making videos with Claude Opus 5.5.

100+ prompts. Free. Completely.

🔗 https://t.co/w9nnEhkDYo

bookmark this page.

then read this ↓

[Jurly @jurlycat]

475 Claude Opus 5.5 video demos in one GitHub repo. 🤯

Motion graphics, explainers, 3D scenes, and games built with HTML, Canvas, SVG, and Three.js.

Includes shared prompts and links to the original posts, so you can pick a demo and try building it yourself.

Repo in the comments.

[beamnxw ./ @beamnxw]

OpenAI Dots is f*cking brilliant for building a 24/7 AI company...

here’s the bigger architecture I’d build around it:

one founder. one dot powered by GPT-6 Astra. 15 specialist jobs connected by work packets

the system has four loops:

BUILD
feedback + support repros → verified evidence → scoped spec → code branch → tested PR

LAUNCH
approved changes → explainers + demo clips + documentation → launch pack

REVENUE
account context → working POC → proposal → follow-up draft
objections and missing features go back into research

OPERATIONS
support triage + invoice drafts + status tracking → one queue of decisions for the founder

the connections are where this gets useful:

a support ticket can become a repro, a patch, updated docs and an answer draft
a finished feature becomes both launch material and proof for the next proposal
a sales objection becomes evidence for the next product decision

give every handoff a file:
→ source references
→ the actual output
→ checks run + open blockers
→ the next job and its exact context

the dot routes the work. specialists return artifacts. you review the decisions and feed corrections into the next task

start with one loop. make it work. connect the next one

save this, then build your company with Dots ⭣

[shikamaru @shikamarutsai]

Anthropic engineer:

"99% of our engineers run swarms of 300+ self-improving agents"

"Now everyone is building agentic graphs"

In a 20-minute session, an Anthropic team member breaks down how graph engineering turns isolated agents into systems that improve themselves

The real setup is Claude running through graph workflows, plan mode, and dynamic orchestration

Agents → Loops → Graphs → Self-Improving Systems

Better than most $300 graph engineering courses

Bookmark and watch the talk

Then read the article below

[Apex @0xApexAi]

KARPATHY WROTE THIS DOCUMENT TO COMPLETELY AUTOMATE YOUR WORK WITH 24/7 AI EMPLOYEE

I was ready to abandon my second brain

manual cross-referencing was destroying my workflow, but finding this exact document opened my eyes to a completely different approach

it is incredibly convenient -> Karpathy's method turns the AI into a full-time maintainer for your Zettelkasten:

> the LLM reads every new source and integrates it into a structured wiki
> the agent runs automated checks to find contradictions across your notes
> your entire vault compounds automatically without you typing a single link

the friction is completely gone. I just feed it raw documents and the agent organizes my entire life

here is the official document from Karpathy explaining the architecture 👇

[slash1s @slash1sol]

An Anthropic engineer packed his whole engineering workflow into agent skills, so anyone can copy it.

Addy Osmani, ex-Google, now works on Claude Code.

His repo turns a coding agent into a senior engineer.

Each skill is a structured workflow: when to use it, the steps, the excuses agents use to skip steps with a rebuttal for each, red flags, and the evidence required before it can call the job done.

25 skills, sorted by phase:

> Define: figure out what to build.
> Plan: break it into small tasks.
> Build: write it test-first.
> Verify: debug and prove it works.
> Review: quality, security, performance.
> Ship: rollout, CI, docs.

Skills also switch on by themselves.
Start designing an API and the API skill kicks in.

Useful if you ship with agents solo, or if you want every agent on your team held to the same bar.

npx skills add addyosmani/agent-skills

The full pipeline is in the article below, repo in the first reply ↓

[Sergii Makarevych @sermakarevich]

An article for everyone who ships, buys, or signs off on software built on large language models (LLMs): engineers, product managers, and the CEO. Written in plain language, from the big picture down to the details. https://t.co/YaP414vR0u

[NO1ennn @N01ennn]

holy sh*t. Sam Altman shows his personal dots setup

one dot. one job. it works the night shift so he doesn't have to:

-> reads everything that came in overnight: email, calendar, Slack, DMs
-> sorts it into urgent vs can wait, based on how he likes to work
-> drafts the replies and sends nothing
-> checks every action against his custom rules: proceed, ask him, or hand it off
-> before his day starts he gets one ping with only the urgent stuff, drafts attached
-> it reaches him in ChatGPT, Slack, Teams or by call
-> whatever he edits or ignores goes back into memory

runs 24/7 on its own cloud computer, through plug-ins he already connected

how i'd copy it:
> one job first
> read-only for a week
> sending always asks
> correct it out loud

included in Pro, Business Premium and Enterprise

give yours the night shift tonight