Zaid Ward

Building multi-agent systems that write to live finance systems. Two-Key: no agent write counts until a second one signs off. Ex-Workday PM. 20+ EPM go-lives.

[Zaid Ward @zaidwardtech]

@BhurkeSiddhesh I'd add a fourth column: stop rate. In my multi-agent builds model spend was about $7 per packet, but 38% of unattended runs stopped, half fixable in tooling. Cost per solved task also needs the orchestrator's spend; mine went unmetered.

[Zaid Ward @zaidwardtech]

@assumebreach Showing reviewers the diff instead of the author's summary fixes the failure I hit most. On a live finance system, what looked like two agents corroborating was really one source relayed through both. Does Quorus log what each reviewer actually read?

[Zaid Ward @zaidwardtech]

@reza_gholiiii Agree on the order. I’d add one thing to step 4: meter the orchestrator. In my multi-agent builds the orchestrator’s own spend went unmetered, so “worth ~15x tokens” was hard to judge honestly. You can’t make that call without the full bill.

[Zaid Ward @zaidwardtech]

@ayazaltai Handoffs are where my multi-agent builds broke too. The worst case: what looked like two agents corroborating was really one source relayed through both. Does structured handoff repair check where a claim came from, or just the handoff format?

[Zaid Ward @zaidwardtech]

@kbhaskar_95 State diffs, yes. Running agents on a live finance system, my checks passed while a board was broken, and only a human-style browser review of the result caught it. I keep an audit ledger of every write outside the agents. Transcripts show intent, state shows harm.

[elvis @omarsar0]

Build specialized harnesses, folks.

This is a banger paper from Google Cloud AI Research on self-evolving agent harnesses.

(bookmark it)

They introduce OO-Harness, which raises AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor.

It also beats Meta-Harness on all three benchmarks they test.

More details below.

Instead of evolving one global harness for every task, an evolution agent builds a class hierarchy of specialized harnesses from execution records.

The design has three parts.

> Subclasses inherit the base harness and override only the methods a task cluster needs
> Design lessons record which changes helped and which ones failed
> A router reads the task and the base run, then picks a subclass or keeps the base result

On Workspace-Bench, the score goes from 25.5% to 39.5%, and on Toolathlon-GYM strict pass goes from 31.2% to 37.5%. Spending extra inference on best-of-K sampling or self-refinement does not reliably help.

Inheritance also improves results. On Workspace-Bench, harnesses built as subclasses score 4.96 points higher than the same evolved code copied into flat classes.

https://t.co/JKZ6y8OXhs

Chat with Paper: https://t.co/Vg9NlOnfEZ

[Zaid Ward @zaidwardtech]

@HaohanWang Curious how the check job handles independence. Running multi-agent builds, my worst failure was corroboration that was really one source relayed through two agents. If agents share memory directly, how do you tell a second opinion from an echo?

[Zaid Ward @zaidwardtech]

@barqers Consent on every call is the right shape. The question I would ask: what happens when the consent check itself errors? In my agent setup the planned MCP proxy fails closed, so no answer means no write. Does this layer default to deny?

[Zaid Ward @zaidwardtech]

@renegadesilicon Agree, and it matches what I saw running agents in a live finance system: lessons filed as prohibitions in a skill rarely fire. Rewriting them as mechanisms, a gate or a check that blocks the write, is what changed behavior. Evals tell you which rules are dead.

[Zaid Ward @zaidwardtech]

@HamelHusain @charles_irl @xeophon Same pattern in my own agent pipeline: checks that passed while blind. My automated checks were green on a broken board, and only a human-style browser review caught it. Now I ask of every check: what would it look like if this passed for the wrong reason?

[Zaid Ward @zaidwardtech]

@SepantaZeighami Splitting knowing from acting matches what I hit running parallel Claude Code agents on a production Pigment tenant. My verifier is read-only with no stake in the outcome. Does the Gatherer ever write, or is it strictly context? That boundary is where my bugs lived.

[Zaid Ward @zaidwardtech]

@ABArkTech_ That question is the whole game. My rule: no agent’s write counts until a second agent countersigns it, and every write lands in an audit ledger. Done is a claim until something with no stake checks it.

[Zaid Ward @zaidwardtech]

@ardabrowski @ghyslainsg @eve @shardara @HamelHusain Same lesson from the agent side. My automated checks passed on a broken dashboard. Only a review that looked at it the way a human user would caught it. The logs hid the failure. The rendered output didn’t

[Zaid Ward @zaidwardtech]

@aiDotEngineer @jxxf Agree on splitting queues. What worked for me: gate by write scope, not agent type. Read-only work runs free. Anything that writes to a production environment needs a second agent with no stake to countersign it.

[Zaid Ward @zaidwardtech]

@random_walker One failure mode I’d want in a field guide: corroboration that isn’t. On a live finance system two of my agents agreed, but one was relaying the other’s output. Do you treat source independence as part of eval design?