Pedro Arantes

Building stuffs using AI

🌳

[Pedro Arantes @arantespp]

@coltfeltes Do you have a system/MCP that implement your rules?

[Pedro Arantes @arantespp]

My rules in building software with AI:

1. Keep all state outside the agent’s conversation. Store each task’s status, blockers, PRs and decisions in a shared record. If an agent did not write it there, treat it as not done.

2. Give each stage one owner. Only the owner of a stage may change a task’s status in that stage, so two agents never act on each other’s stale view.

3. Separate the author from the reviewer. Review every change in a fresh agent session that did not write it and is not allowed to fix it.

4. Close tasks on observed results. Have the implementer state what to check, where and when. A separate check confirms the effect in production before the task closes.

5. Cap the automatic fix attempts. After a set number of failed rounds, hand the change to a person.

6. Let reversibility set the agent’s freedom. Agents act alone on what can be undone and noticed in time. A person approves the rest.

7. Never let an agent guess an open decision. Record it as a question for the person who decides, and block only the work that depends on it.

8. Order the work by the cost of waiting, read from its stage. A broken main branch first, then urgent items, then high-priority bugs, then deploys nobody has verified, then due follow-ups, then open PRs, and new work last.

9. Limit the work in review and release. When that queue is full, agents stop starting new tasks and help finish the open ones.

10. Enforce merge rules in the code host. Use branch protection and required checks, and require a person’s approval for any change to the checks or the agent instructions.

11. Start every agent session from a schedule. A scheduled job wakes the agents, and each session reads the current state from scratch.

12. Write down why each rule exists. A rule you cannot justify with a principle is one to remove.

[Szymon Rybczak @SzymonRybczak]

what's the best way to build software factories right now?

we're writing a guide on verifying what the agents actually ship, need to learn more about people's setups

[Pedro Arantes @arantespp]

@Raymond_0716 If you're interested in knowing the MCP I created to handle the shared record, I want to share:

https://t.co/dudprC7IrX

[Pedro Arantes @arantespp]

I want to present you my agentic workflow.

[Pedro Arantes @arantespp]

@anikket_b I do keep them apart. Each task says upfront what integrating leaves, a release step or a check with what to observe and when.

While the release is pending it’s blocked, while the check is pending it’s on the calendar, and it only closes with what was observed in production.

[Pedro Arantes @arantespp]

I think if you opt to not review your code anymore, you must have observability to watch what you ship on production.

That’s what I did with my agents. A task only closes when:

- I see its effect in e2e tests
- live validation works
- production logs shows what I expect

[Dan ⚡️ @d4m1n]

I don't see the point to review AI generated code anymore.

AI agents are the new compiler.

We don't review bytecode. We just trust the output of the compiler.
I don't think we should treat AI output differently.

However, the authoring layer cannot be markdown. I feel like there's a paradigm just around the corner that we have missed so far. Maybe a new "language", which won't look like a programming language at all.

[Pedro Arantes @arantespp]

Yesterday, I finished the order my agents pick up work. New features come last.

The main principle: Do first the job that is most expensive to leave waiting (cost of delay).

The problem: agents start work faster than anyone can finish it. If they pick new features, half-done work queue increases and value doesn't reache users.

The rule comes from Reinertsen: order work by cost of delay. My agents read it from the stage the work is in. Every tick works in this order:

- main branch red: fix it first, it holds every change in the repo
- anything a person marked very high
- bugs marked high
- merged work nobody has validated in production
- waits that came due
- open PRs: conflicts, fix rounds, reviews
- new work

A small example: main is red and 6 PRs wait on it. One repair frees these six changes. A new feature started in that hour frees nothing.

The priority labels only order work inside a stage. A high-priority feature still waits behind an unmerged low-priority PR. Only “very high” jumps stages.

[Pedro Arantes @arantespp]

I tell clients they can build it themselves with AI.

The question I ask back: do you want to spend your energy on architecture, security, tests and servers, or on your product and your customers?

Whoever picks the second one will continue hiring a software engineer.

[Pedro Arantes @arantespp]

I want to present you my agentic workflow.

[Pedro Arantes @arantespp]

@JessePeplinski Yes, I build flow

https://t.co/fKCNt8CdiG

[Pedro Arantes @arantespp]

One task graph every agent reads and writes over MCP, and a method for it, Get Things Flowing.

1. The graph computes what can run in parallel. Wave 1 = how many agents you can start now.

2. Each agent takes one task, with its own key.

3. Done means written back: status, outcome, PR.

4. An agent never makes your call. An open choice becomes a decision waiting on you.

5. Side quests go to the inbox, not into the code.

6. You answer once, and the work is ready for the next agent.

[Pedro Arantes @arantespp]

I tell them yes, they can. But they own the rest:

- define the architecture
- review security
- maintain it
- add observability
- write tests
- run the deploy and the servers

If they’d rather spend their energy there than on the product, their customers and their brand, they should do it themselves with AI.

[Fadi Hares @fadihares95]

You use AI to finish a client’s app in 2 days.

The client asks why they shouldn’t just use AI themselves.

What’s your answer?

[Pedro Arantes @arantespp]

Check https://t.co/U1N3LRKxe1

[Pedro Arantes @arantespp]

I decided to solve everything around the model.

An agent that serves customers needs memory of each one, search over your documents, a WhatsApp channel, a schedule, a rule for which actions need a person, and a record of what every run cost.

I’m building naturali to be that infrastructure. Each piece is a module behind one API: agents, knowledge, memories, channels, workflows, guardrails, evaluations. An AI app becomes a set of API calls.

The first real test is inside OneClick Ads. We’re using naturali’s knowledge and memories to build a second brain of our customers, so the ad recommendations know each account.

What this buys a builder:

- you start from the part only you can build
- every run is traced with its cost from day one
- the SDK, the CLI and the MCP server share the same API, so you can build by talking to Claude

[Pedro Arantes @arantespp]

In naturali, you can attach a guardrail on agent's tools. The guardrail is a versioned rule document, and it sorts the call into four tiers:

A: read-only, it runs
B: runs only if its guard passes
C: waits for a person to approve
D: blocked, and the model is told so

[Pedro Arantes @arantespp]

@levelsio @X Thanks @levelsio !

[Pedro Arantes @arantespp]

I taught my agents to estimate cost of delay. They read it from the stage the work is in.

The stage gives a ranking that holds for every project:

- a defect in production hurts users every hour
- an open PR gathers conflicts
- a deploy nobody validated lets a regression run
- a new task loses little by waiting one more round

So an agent can flag a critical bug and move it to the front without any math. The only override is a person marking something top priority, because that is their own word that waiting costs most.

It’s the principle behind the rule I: validations and merges first, new work last.

[Pedro Arantes @arantespp]

If you are about to put agents into your delivery process, my advice is the same thing I did: before you write the first rule, write down why. https://t.co/QC68w4YF9V

[Pedro Arantes @arantespp]

I set one priority rule with my team: post-deploy validations and merges come before any new task.

The rule is simple: when you can’t do everything at once, do first the job that is most expensive to leave waiting.

[Pedro Arantes @arantespp]

@msdevelx Mutation tests is a new thing for me. I'm still learning more about them.

[Pedro Arantes @arantespp]

My experience with tests and AI:

Requiring 100% test coverage made my agents' test quality worsen.

I set that target in my agent workflow months ago. CI got slower on every release, implementations took longer, and the agent started writing tests that existed only to reach the number. The suite stayed green and proved little.

This is Goodhart’s Law: when a measure becomes a target, it stops being a good measure. The cheapest path to 100% for an agent is a test that asserts almost nothing.

The principle I cite most often: the author never verifies its own work. An agent judging its own change will turn the checks green whether or not the problem is solved.

So I dropped the coverage target and moved verification up a level:

- end-to-end tests on the behavior a user would notice
- observability across the whole system, to see the effect after it ships

What it buys:

- CI time goes to tests that catch real breakage
- green starts to mean the change actually worked

Same principle as before: the author never verifies its own work, and a coverage number written by the agent is still the agent’s verdict.

[Pedro Arantes @arantespp]

@kseniam0s I build in Brazil, where most investors only put money in after PMF is proven. I wish I knew that before so I didn't spend much energy in meetings with VCs.

[Pedro Arantes @arantespp]

@prpatel05 You're falling behind because there are 100x devs that just press tab.