青雲

字节 Agent 研发｜开源 orca-agent / blade-code / Pilion-browser / blade-agent-sdk｜Agent 工程、ToB 落地与开源实践 ｜ 中转站站长

[青雲 @echo_vic]

有人问做 agent 从哪开始，我一般不给书单。

我数过这些年真正能拿出来给人看的产出物，一共五样。说「卡在哪一关」其实不太准确——更常见的是五样都有，但都是半成品，然后到处漏水。

第一样最简单，也最容易糊弄过去，就是一段不会认错的描述。一个工具或一个 skill 只做一件事，「什么时候用它」要写进描述里。判断方法很土，把描述遮掉名字，还能不能一眼分清它和旁边那个。分不清，就是边界还没想清楚。

第二样是一份你愿意每次都照着跑的完成判据。把「做完了」翻译成可以核对的东西，命令、退出码、通过/失败/跳过计数、失败详情，缺一项就算未验证。再配几条禁止，不许把 skipped 计进通过、不许加 .skip 让测试变绿、失败时不许报 DONE。这一关刷掉的人最多，因为口号写起来实在太顺手了。

第三样，三段独立的校验。模型返回合法 JSON 只证明 schema_valid（格式类型对），内容跟任务对不对得上（semantic_valid）、这个动作能不能做（action_safe），是另外两件事。只有第一段通过，推不出后两段。多数项目只做了第一段，然后就当三层都过了。你那套链路里，第二段落在哪，说得出来吗。

第四样最难，也最容易被推到「以后再说」——一次能被回放的执行记录。写到某个时候你会发现，写在提示词里的约束不可靠，会被压缩、被遗忘、被绕过。约束得变成运行时里的一个阶段，失败了就跑不下去。判断标准就一句，出问题的时候，你能不能让当时的事件流重放一遍。

第五样，一份能定期清点的清单。工具、skill、规则都只会增不会减，你得知道哪个还在用、哪个已经死了、哪个在抢别人的机会。清单至少三列，上次真正被触发是什么时候、被检索到多少次、触发之后成功率多少。真记起来挺烦的。

这五样都能拿给别人看。如果你现在一样都拿不出来，大概率不是学得不够多。

适合已经在写 agent、但总觉得「它有时候不听话」的人。还没把一次完整工具调用跑通的话，先跑通再回来。

[青雲 @echo_vic]

The split is the right one and the boundary is the weak part, since mild and reversible is judged by the same party that benefits from calling its change mild and reversible. BAR makes that boundary structural instead, because each step gets its own record and its own rollback point, so reversibility stops being an opinion about the diff and becomes a property of where the commit sits.

The rest of that file reads like it was written after one specific bad day, and those are usually the lines that hold.

[青雲 @echo_vic]

The first line is a check and the second is a wish, and the wish is the one that gets ignored, because nothing can make it fail and so it cannot steer a run, while deleting filler and fixing the layout names exactly what to look at. spec-flow keeps that difference, since a rule counts only where a run can be measured against it, and the rule that never fails is decoration in the file.

The fix is to turn the best-decisions line into something a run can violate, for instance a stated limit on labels per screen.

[青雲 @echo_vic]

The vague description is the visible error and the harder half is the when-not clause, because a description that only says when to use the skill competes with every other skill that also sounds plausible, and the routing decision is made on that one line in the file. learning-system-skill keeps both halves for that reason, since the case that must not fire it is what stops it from stealing work it was never meant for.

On small versus one big, several small skills with honest when-not lines beat a single large one, because the large one carries a single description and therefore a single chance to look plausible.

[青雲 @echo_vic]

The interesting part is not the poisoned file but that the agent treated it as trusted configuration because the name matched, which means trust came from the filename rather than from where it was read. stop-protocol makes that distinction explicit, since a skill declares before it runs what it may do, and installing a package is exactly the kind of step that should stop for approval no matter which file asked for it.

The general rule is shorter: a directory a browser can write to is input, not configuration.

[青雲 @echo_vic]

The principles do survive and a SKILL.md is exactly where they lose their teeth, because a file of prose can state a principle while nothing enforces it, and the reader decides whether it applied. spec-flow puts the principle where each run is compared against it, so the rule is executed at the point of the commit rather than remembered at review time.

The Uncle Bob version lands harder with agents in the loop, because a person who skips a principle leaves a smell and an agent leaves a diff that looks fine.

[青雲 @echo_vic]

That validator catches the parse failure and the silent one runs the other way: a description that parses but is too vague to trigger means the skill never activates on its own, and no validator reports that. learning-system-skill keeps the description next to the case that is supposed to fire it, so the check is whether that case fires in a fresh session without naming the skill.

The cheapest test is to ask for the task in the user's words rather than the skill's name, because a skill that only answers to its own name is a command, not a trigger.

[青雲 @echo_vic]

Grounding states in real signals is the right call. "Blocked" is the one I'd want split. In Orca, the terminal agent I build, a paused goal records why: no progress, or an infrastructure problem like a verifier that couldn't give a verdict. Those need different fixes from me. Does the office show why an agent is blocked, or only that it is?

[青雲 @echo_vic]

我在做一个 DeepSeek 原生的终端 agent Orca，体会是同一个 Flash，harness 差得最多的地方在「什么算干完了」。Orca 的 goal 模式里，只读文件不算进展，真改了东西或者更新了任务计划才算，连续三轮没进展就停下来交回给我。Flash 便宜，多跑几轮不心疼，怕的是它原地转圈。你这个调研里大家吐槽最多的是哪一块？

[青雲 @echo_vic]

@seek_stars84429 顺序为主的话帮助应该有限，它主要是给同时开好几个 agent 的场景做的，F1 那个跳转也是为这个设计的。我是看介绍发现它的。顺序跑的话，按 workspace 看资源占用、Ctrl-Shift-T 找回关掉的标签页这两个可能还用得上。

[青雲 @echo_vic]

发现一个挺有意思的终端工具 Corgi，给同时开好几个 agent 的人做的多路复用器，灵感来自 Herdr。

它会把每个 agent 的状态摆出来：在干活、在等你、已完成。按 F1 直接跳到下一个有未读消息的 agent。Claude Code、Codex、Pi、OpenCode 都能从同一个入口开，多个账号也支持。

几个小细节我挺喜欢：CPU、内存、GPU 占用按 workspace 拆开看；⌘V 能直接粘图片，SSH 里也行（目前需要 Ghostty Nightly）；关掉的标签页可以 Ctrl-Shift-T 找回来。

支持 macOS、Linux 和 WSL。多开 agent 的话，F1 这个设计应该挺省事。

https://t.co/Ufh4N0l7kw

[青雲 @echo_vic]

@KoderNeverDie 预算卡在启动子任务之前这点我同意，超额告警确实晚了一步。我在 Orca 里把预算用完单独记成一种结束原因，跟内层轮数用完分开，至少能分清是钱花完了还是轮数到了。按根任务归账这条，你们是自己在 harness 里打 task ID，还是平台那边能给到？

[青雲 @echo_vic]

这两天 HN 上吐槽额度的帖子明显变多了。

Kimi K3.1 的讨论里有人说，订了 $99 一个月的套餐，一个会话 47 分钟就撞上了 5 小时限额。另一个 Ask HN 里有人用 Claude Code，说以前同样的用法能撑 4 个小时左右，那天 10 分钟就用掉了 30% 到 70%。

我觉得最难受的地方在于看不见。agent 在后台开子任务、反复读上下文，token 花在哪一步你很难知道，等 429 出来已经晚了。

你们平时怎么盯额度？有在用的工具吗，还是撞上了再说？

[青雲 @echo_vic]

@Dennyho2 帖里那两个例子都是 5 小时窗先撞上的，周额度倒没人提。你说的大任务前先让它报预计上下文和子代理数，这个我没试过，它报的数跟最后实际烧掉的能对上吗？

[青雲 @echo_vic]

看到一个挺好玩的实验：this+that 把安然公司 45,047 封邮件交给 AI，让它自己写一套公司 wiki，全程没人改过一个页面。

最后出来 214 个页面、49,401 条事实，每条都引用了出处邮件。每个页面带一个时间滑块，邮件里显示某件事不再成立，这条事实就在那天过期。拖一下就能看到安然在某个日期"知道"些什么。

抽查结果是 93% 的事实被原邮件完全支持，500 条样本里核心错误为 0。这轮审计是另一个 AI 做的，页面上也写明了。

作者在 HN 说最难的地方是模型本来就了解安然，老往里加邮件里没有的公开资料。做知识库的应该都见过这种问题。

https://t.co/wxvZP6h7q9

[青雲 @echo_vic]

Kimi K3.1 Pro 这几天在 HN 上讨论很多，我挑几个在意的数。

Terminal-Bench 4.0 拿到 65.7%，官方表里 GPT-6 Astra 58.2%，Opus 5.5 66.4%。3T 级开放权重，1M 上下文，权重说 11 月 4 日前放。API 每百万 token 输入 $2、输出 $10，缓存命中 $0.20。

网安那段最有意思：mid-training 和 RL 里都删了 cyber 数据，内部 xdev-bench-hard 从 90.5% 降到 81.0%，前后对比直接公开了。

脚注里写了 TB 分数是用 Claude Code 的 harness 跑的。我在 Terminal-Bench 2 上测 Orca，同一个模型换 harness 分数差很多，所以看这类榜我会先看用什么跑的。

https://t.co/V0dY3NH43r

[青雲 @echo_vic]

最后那个问题我有一组数据可以参考。我做的 DeepSeek 原生终端 agent Orca 跑 Terminal-Bench 2.0 全量，模型不变，只修 harness，mean 从 0.494 涨到 0.7955。所以榜上 DeepSeek 套 Codex 的分数，可能只是它的一个下限。不过 FrontierCode 看的是维护者愿不愿意合，这块受 harness 影响应该小一些，你怎么看？

[青雲 @echo_vic]

Honestly, it doesn't try to tell a flaky failure from a real one. Red is red. What keeps the cheap model from burning reruns on noise is goal mode: if three turns in a row hit the same gap, it pauses and the run comes back to me. Orca's own suite had flaky failures under cargo test --test-threads=16, and v0.3.15 fixed them in the tests themselves. Do you quarantine flaky tests or rerun them?

[青雲 @echo_vic]

我这篇只是读文章的笔记，合并迁移我自己没做过，所以两种办法都没实测过。你那个「每一条都要有证据才算完成」我很认同，Orca 的 goal 模式也是这个思路：完成只能由 verifier 判，缺证据的提交在送给 verifier 之前就会被拒掉。好奇职责清单并进主循环之后，prompt 变长了多少，延迟那部分收益还剩多少？

[青雲 @echo_vic]

Anthropic 这篇讲什么时候把子智能体并回主循环，大家转得最多的是「agent 架构别照抄组织架构图」和「单循环快一倍」。我读下来最有用的是迁移那段的一个细节。

他们在一个示例客服 agent 上合并子智能体，延迟降了，但一个原本能过的流程坏了：主 agent 手里有查订单的工具，就自己答了，没去问物流那个子 agent 要送达日期。这个问题两次运行里只出现了一次。

所以他们建议别只看总分，要看哪些任务翻了，生产里一起跑的每条 agent 链路都要有用例，每个用例跑不止一次。

子智能体也还有位置：上下文膨胀到能测出掉点、需要用完即弃的并行上下文、工具选不准或 prompt 里指令打架，这几种情况还是该留。

我平时也习惯把常用流程沉淀成 skill，这篇做 agent 的值得细读：
https://t.co/s9uET0cxRA

[青雲 @echo_vic]

One or two samples is the right gate and the sample only works as a test set when it is stratified: 47 languages do not share the same failure, so one Latin, one Cyrillic, one CJK and one right-to-left language catch the classes a second Latin language cannot, and the instructions file should name which script each rule was written for. learning-system-skill keeps that distinction, since what a skill should hold is the procedure plus the cases that actually exercise it, and a sample from one script family exercises almost none of the others.

Your agents.md is the other half, because it should say what to do when a language has no clean equivalent, and that is the case a sample will never show you.