Jayxu.net

Build AI Agent for enterprise. Share AI best practice.

[Jayxu.net @xubujie]

@adxtyahq A practical split: capability evals ask “can it do the task?”; regression evals ask “did we break known behavior?”; operational evals ask “can we detect and recover from failure?” Teams often stop at capability scores and miss the last two.

[Jayxu.net @xubujie]

@PawelHuryn A useful addition for PMs is “artifact”: the durable output an agent leaves behind—a plan, decision, evidence bundle, or patch. If a loop produces no inspectable artifact, it is difficult to evaluate, hand off, or recover.

[Jayxu.net @xubujie]

@RaghavChamadiya This is the right interview shape. I would add one more step: introduce a nearby failure after the candidate accepts the patch. Can they predict it, update the test, and explain the invariant? Agent fluency matters, but causal understanding makes the work maintainable.

[Jayxu.net @xubujie]

@dorukkavcioglu The interesting shift is not AI generating video; it is making editing addressable through structured operations. If every cut, asset, and effect is represented as replayable commands, agents can iterate, diff, and recover instead of merely clicking a UI.

[Jayxu.net @xubujie]

@pidotdev Four tools are enough if their contracts are sharp. The real leverage is observable execution: explicit working directory, scoped permissions, exit codes, diffs, and test results. Add a new tool only when the primitives cannot be composed safely.

[Jayxu.net @xubujie]

@brenzhills Keep identity and state durable; keep compute ephemeral. Wake agents on events, checkpoint after tool calls, and measure cost per completed task—not uptime. Most agents should be resumable jobs, not tiny always-on servers.

[Jayxu.net @xubujie]

@gregisenberg Agent commerce needs more than payments: machine-readable pricing, scoped identity, per-task spending limits, idempotent purchase APIs, and signed receipts. Without those controls, an agent is not a customer—it is an unbounded procurement bot.

[Jayxu.net @xubujie]

@wquguru 我更倾向于按风险分层，而不是简单分成 vibe coding 和正式工程：低风险改动可以让 agent 自主完成；高风险改动必须附可复现测试、diff 审查和回滚方案。AI 生成代码不是问题，缺少验证闭环才会变成屎山。

[Jayxu.net @xubujie]

A reliable coding agent should leave three artifacts:

1. Plan — what it intends to change
2. Evidence — tests, diffs, and observed behavior
3. Handoff — remaining risks and next owner

If it cannot produce all three, it is still a demo—not an engineering system.

[Jayxu.net @xubujie]

@BkashJosi The leverage comes from reusable skills with explicit inputs, trusted sources, and verification steps. Without those contracts, an 'agent operating system' is still just a very busy prompt.

[Jayxu.net @xubujie]

@svpino Code review shouldn't disappear; it should move up a level: inspect intent, invariants, tests, security boundaries, and production behavior. Reading every generated line isn't the same as owning the outcome.

[Jayxu.net @xubujie]

@de1lymoon A useful progression: autocomplete → repository context → tool use → execution feedback → eval-backed iteration. The key shift is making each failure observable and recoverable, not just generating more code.

[Jayxu.net @xubujie]

@bibryam Exactly. Start with the cheapest reliable classifier, preserve structured text, and escalate only ambiguous pages to OCR. This is agent-friendly design: routing, fallback, and measurable failure modes.

[Jayxu.net @xubujie]

@wuweiweiwu @posthog PR volume is a useful signal only alongside outcome metrics: regressions, cycle time to useful change, incident rate, and reviewer load. More merged code isn't automatically more shipped value.

[Jayxu.net @xubujie]

@simonAI_X The missing ninth layer is ownership: who approves side effects, who can stop the agent, and how failures feed the next eval set. Reliability begins where the architecture diagram meets operations.

[Jayxu.net @xubujie]

@enunomaduro Human guidance scales when you encode it: architecture constraints, representative examples, failing test cases, and explicit acceptance criteria. Otherwise the agent optimizes for a plausible diff, not maintainability.

[Jayxu.net @xubujie]

@tom_doerr Isolation is only step one. Parallel agents also need an explicit ownership map, shared acceptance criteria, and an integration gate—otherwise two green CI runs can still create one broken architecture.

[Jayxu.net @xubujie]

Most AI agent failures aren't model failures. They're missing boundaries.

Before production:
1. Clear task + success criteria
2. Least-privilege tools
3. Observable execution
4. Evals on real failure modes
5. Human-owned rollback

Demos show capability. Operations earn trust.

[Jayxu.net @xubujie]

@codeo4 图给codex做就好了呀

[Jayxu.net @xubujie]

@AISuperDomain @OpenAI @thsottiaux 真的张口就来，估计文案都是ai写的