AI agents will soon be able to create whatever software a job requires.
The difficult question is no longer just whether they can build it. It is whether we can bound what they do, verify the outcome and keep humans in authority.
I write about that, along with Bitcoin and systems thinking.
@AgenticAIFdn@goose_oss For most missions, telemetry is the flight recorder, not the control system. It can explain a destructive action after the event, but it does not bound one beforehand. For consequential tools, I’d want both: a pre-action authority check and a post-action evidence trail.
@lopp This is an assurance-boundary problem. Reviewing firmware source is not reviewing what users run if the build chain and submodules sit outside the evidence set. “Audited” should name the boundary, or confidence in one component gets mistaken for assurance of the whole.
@athyuttamre An agent needs to distinguish “tell me what would happen”, “prepare it” and “do it now”, especially when an MCP has side effects. Do we need a boundary with the client, the MCP, or both?
@GoKiteAI Putting the rules in one SKILL.md makes intent inspectable. The harder question is whether execution can prove which version the agent read, which commands it issued and whether it stayed within those rules. Does the passport bind instructions to an audit trail?
@nnennahacks Useful eval. I wonder if the harder test is behavioural rather than textual: does the generated AGENTS.md actually change an agent's decisions when instructions conflict, and does that effect hold across tasks and models?
The Coldcard incident is a sharp assurance lesson. The intended hardware RNG was present in the firmware, and review had verified its presence. But the seed-generation path resolved to a different implementation.
That is the gap between control existence and control effectiveness. Seeing a safeguard in the design, build or policy is not enough. You need evidence that the real execution passed through it.
Agentic systems will multiply this problem. Policies and guardrails may be present while generated workflows take an unexpected path. Assurance has to follow the mission from intent, through execution, to independently verified outcome.
@ZynxBTC The irony right now is that many of the same maxi/zealot voices that heavily criticised Ledger for requiring trust in closed-source code are now dealing with (or watching) a serious failure in one of the most respected fully open-source devices.
@FabricFND@GoKiteAI Identity and spending limits tell us who the agent is and how much it may spend. They don't yet prove the purchase satisfied the mission. Did it buy the right service, on permitted terms, and who accepts the evidence that the outcome was achieved?
@NousResearch@blocks Putting humans and agents in the same workspace makes their activity visible, but visibility isn’t authority. I’d want the workspace to make three things obvious: who commissioned the work, what an agent may change, and who verifies the outcome before it becomes accepted state.
Software may become disposable. Assurance can’t.
If an agent writes code for one job and throws it away afterwards, we still need to know what it changed, what authority it had and whether the result was independently verified.
The code can disappear. The evidence can’t.
@AgenticAIFdn@AdoraNwodo Contracts are the right direction, but registration and ownership only get you so far. Once agents can act, the contract also needs to say what they may change, what evidence must survive, who independently verifies the outcome, and when authority returns to a human.
I’ve been thinking about what happens when AI agents can write software whenever they need it.
It might become a bit like a satnav route. You don’t ask how the route was calculated. You just ask to get to London. The route is useful, but it isn’t the thing you value.
Maybe software starts to work like that too: created for a job, used, then thrown away.
What matters is the mission, the rules around it, and knowing whether it actually worked.
Keep Bitcoin’s base layer simple, neutral, scarce, and secure. Keep innovation at the edges, where adoption is voluntary and failure is local.
Protocol changes must be rare, conservative, and driven by necessity, not ambition.
Defend Bitcoin’s constitution. Defend the future.