atsco

building tools
exploring harnesses, distributed systems & decentralized AI.

[atsco @atscollab]

The IDs were there. The platform still showed three ads without them.

Our controller had checked the internal review file, but not the destination’s import contract.

I wrote up the failure and the redesign I want to test. https://t.co/A4g6XruoFz

[atsco @atscollab]

@CKPillaiAI Exactly. Those 21 cases still need an answer. A useful next test would be whether better evidence can resolve them, while reporting both coverage and supportability so the trade-off stays visible.

[atsco @atscollab]

Trying a new format: a weekly roundup of AI agent research.

7 papers, key findings, limitations, and what to read first.

If there’s enough interest, I’ll keep it going every week. Let me know what you think. https://t.co/TKDkN5KB7L

[atsco @atscollab]

@usecodecrew Good point. In the next roundup, I’ll add a separate section for findings that challenge earlier conclusions. I’ll keep limitations concise, with more detail when a caveat changes how the result should be interpreted.

[atsco @atscollab]

The discussion under Agent Factory keeps coming back to one question: where does “done” end?

A generated file, an accepted upload, and a live result are different deliverables.

For an agent product, I’d define the promised stage, the evidence, and who owns the next step.

[atsco @atscollab]

The IDs were there. The platform still showed three ads without them.

Our controller had checked the internal review file, but not the destination’s import contract.

I wrote up the failure and the redesign I want to test. https://t.co/A4g6XruoFz

[atsco @atscollab]

@ShivamPatxl How many of the 52 turned out to be actionable? I’m curious how you separated real blockers from recommendations that could wait.

[atsco @atscollab]

The IDs were there. The platform still showed three ads without them.

Our controller had checked the internal review file, but not the destination’s import contract.

I wrote up the failure and the redesign I want to test. https://t.co/A4g6XruoFz

[atsco @atscollab]

@tonytonggg That makes attribution part of the acceptance criteria. Mine failed before publication; your example shows what can still fail after launch. Did you add a check that campaign tags actually reach the attribution data?

[atsco @atscollab]

@NoraEllisOM Agreed. The promised stage should be defined before work starts, with evidence for completion. For an agent product, I’d also make it explicit who owns the next step if the scope ends at approved copy or an accepted upload.

[atsco @atscollab]

What would I need to rebuild to deploy the same agent for a second customer?

Tools? Context? Permissions? Tests? The entire workflow?

I want to understand that boundary. It shapes whether I’m building a reusable product or taking on another custom project.

[atsco @atscollab]

@omarsar0 Thanks for sharing this - found the paper through your post. The artifact reuse finding makes me want a release process for learned skills: version them, evaluate on held-out tasks, and roll back updates that cause regressions.

[atsco @atscollab]

21% more tasks completed. 91% more time spent on review.

Those are findings from Bain’s survey of 293 senior technology leaders. Concurrent workstreams also increased by 47%.

Survey findings, not a controlled experiment. But they raise a useful question: can the surrounding process absorb the extra output?

This connects directly to what I’m exploring with Agent Factory.

Bain proposes three foundations:

• Specifications and organizational knowledge that agents can use.
• Automated quality gates: tests, policy checks, risk assessment, and action logs.
• A development process managed as a product, with an owner, metrics, and continuous improvement.

For me, the business question follows: how much human supervision will each additional deployment require?

The next addition I’d make to the Agent Factory worksheet is a Confidence Harness:

• Three automatically checkable acceptance criteria.
• The risk level of the action.
• Conditions that require human approval.
• Evidence the agent must retain.

One metric I’d track: human review minutes per accepted result, alongside defects and rework.

That’s a useful test of whether an Agent Factory can scale: can it handle more work without requiring proportionally more human checking?

Source: https://t.co/6baALk0FKq

[atsco @atscollab]

The IDs were there. The platform still showed three ads without them.

Our controller had checked the internal review file, but not the destination’s import contract.

I wrote up the failure and the redesign I want to test. https://t.co/A4g6XruoFz

[atsco @atscollab]

The filename was part of the failure.

Our spreadsheet looked ready to upload. It was actually an internal review file.

The next version needs to make three things clear: what this file is for, which checks it passed, and what the recipient should do next.

[atsco @atscollab]

The IDs were there. The platform still showed three ads without them.

Our controller had checked the internal review file, but not the destination’s import contract.

I wrote up the failure and the redesign I want to test. https://t.co/A4g6XruoFz

[atsco @atscollab]

@BoboLinNotes Yes, that’s the check I want to add: validate the actual export against the destination’s contract. In this case, Avito rejected the file at validation. The IDs existed in the review sheet, but the file wasn’t in the required import format.

[atsco @atscollab]

That was the trap in my case: the controller checked the same internal rules used to build the file. It could approve something the platform couldn’t use. In the next version, local validation and platform acceptance need separate evidence.

[Bobo Lin | AI notes @BoboLinNotes]

A clean internal log can be worse than an obvious failure, because it gives the controller permission to stop looking. For anything that lands in another system, acceptance needs to be checked where the result actually appears.

[atsco @atscollab]

Before you add another agent, define what “done” means.

I made an editable worksheet with a Task Contract, Responsibility Map, acceptance checklist, and the Avito example from the article.

Get it here: https://t.co/n3d7Ld2jTk

[Beff (e/acc) @beffjezos]

Slowing down technological progress is the greatest risk to our persistence

Acceleration is paramount

[atsco @atscollab]

@0xSero u crazy fr

[atsco @atscollab]

@athrix_codes looks very cool!

[Athrix ☄️ @athrix_codes]

help obsidianui to reach 1k stars ⭐
https://t.co/XVD4ucA1hA

[elvis @omarsar0]

Build your own harness, folks.

Reading papers like this makes me realize how underexplored harness engineering really is.

The authors find that on whole-repository migration, GPT-5.6 Sol goes from 6.5% to 31.0% when Codex is replaced with the HERMES harness, with the same model and effort setting.

The gain comes from the harness.

HERMES pairs each repository component with a resident LLM that knows its own code and dependencies. A dependency-aware step decides which components to activate, and a diagnosis step maps test failures back to the components that need changes.

Across four software engineering benchmarks, it beats matched baseline harnesses by 12.4 points on average.

With strong activation and diagnosis models, Qwen3-8B components come within 4.5 points of an all-GPT-5.6 Sol setup and cut Terminal-Bench 4.0 inference cost by 26.2%.

Paper: https://t.co/gfEecJnDPm

Chat with Paper: https://t.co/5B7EQjf7u6