Adil Patel

Engineer who ships. CTO of A3Q, ex-AWS. Pro-AI, anti-fake-productivity. Retired UAB hockey bench warmer.

[Adil Patel @TheRealAdilP]

I stopped letting my agents grade their own tests and built a CLI that does it. Subagents checking their work after the last edit went from 42% to 63%

Here's how it works and how to set it up in your repo.

The problem. Rules in CLAUDE.md get followed sometimes and ignored other times. That's fine when I'm watching. It's not fine for subagents running unattended, and an agent checking its own work is a student grading their own test.

So I built a3q for my codebase. The agent still takes the test, but a tool grades it and another stops it from hiding a failing result.

1. a3q check. It reads the diff, picks the tests and type checks it touches, runs them and prints counts, failing test names and first errors. It exits non-zero on any failure and a run with zero tests fails. The agent just reports what it said. CI runs the same check.

2. a3q prove-red. It breaks one line, requires a test to fail and always restores the file. A red that's only a compile error doesn't count. This proves a test actually guards something.

3. Hooks. They refuse known-bad tool calls and the error names the fix. Pipe test output into tail or grep and it gets blocked with "run a3q check and save the full output to a log". Of 320 pipe blocks, agents switched to a3q check 46% of the time and saved a log 23% of the time. 4% tried another wrapper. None retried the same command.

Same idea for prod. It's read-only because the tool refuses writes before it connects, not because the agent promised.

The numbers. 9,600 sessions, Sept 10 to Oct 3 vs Oct 4 to 28. Subagents that edited and then verified went from 51% to 69%.

What it doesn't show. Top-level sessions I'm watching barely moved, 81% to 77%. Last check passing fell from 95% to 84%, probably because failures aren't hidden anymore, but I can't prove that yet. And it's built for my codebase and my opinions.

To copy it:
1. Find the rule your agent breaks most
2. Put test running behind one script that picks tests, prints counts and fails on zero tests
3. Add a hook that blocks the bad command and prints the fix

Send this to Claude Code 👇
"Write a Claude Code PreToolUse hook that blocks any Bash command piping test output into tail, head or grep. Exit 2 and print: Blocked, piping hides failures. Run ./scripts/check and save full output to logs/test.log"

Every time an agent breaks the same rule twice I stop adding it to CLAUDE.md and write a check instead.

What rule does your agent break most?

[Adil Patel @TheRealAdilP]

@steipete Is the hooks layer the part you care most about owning? Mine block agents from piping test output and name the fix

[Adil Patel @TheRealAdilP]

@emollick A lot of that gap is the harness and adding a check command and hooks took my subagents from 42% to 63% checking after their last edit

[Adil Patel @TheRealAdilP]

@jeremybernier How do you find signal for if the person you are interviewing can dive deep, have enough technical knowledge to make good decisions. You can get that from a 1-2 day trial either.

[Adil Patel @TheRealAdilP]

@Joshua_WD Back to regular old fashioned code 😆

[Adil Patel @TheRealAdilP]

I stopped letting my agents grade their own tests and built a CLI that does it. Subagents checking their work after the last edit went from 42% to 63%

Here's how it works and how to set it up in your repo.

The problem. Rules in CLAUDE.md get followed sometimes and ignored other times. That's fine when I'm watching. It's not fine for subagents running unattended, and an agent checking its own work is a student grading their own test.

So I built a3q for my codebase. The agent still takes the test, but a tool grades it and another stops it from hiding a failing result.

1. a3q check. It reads the diff, picks the tests and type checks it touches, runs them and prints counts, failing test names and first errors. It exits non-zero on any failure and a run with zero tests fails. The agent just reports what it said. CI runs the same check.

2. a3q prove-red. It breaks one line, requires a test to fail and always restores the file. A red that's only a compile error doesn't count. This proves a test actually guards something.

3. Hooks. They refuse known-bad tool calls and the error names the fix. Pipe test output into tail or grep and it gets blocked with "run a3q check and save the full output to a log". Of 320 pipe blocks, agents switched to a3q check 46% of the time and saved a log 23% of the time. 4% tried another wrapper. None retried the same command.

Same idea for prod. It's read-only because the tool refuses writes before it connects, not because the agent promised.

The numbers. 9,600 sessions, Sept 10 to Oct 3 vs Oct 4 to 28. Subagents that edited and then verified went from 51% to 69%.

What it doesn't show. Top-level sessions I'm watching barely moved, 81% to 77%. Last check passing fell from 95% to 84%, probably because failures aren't hidden anymore, but I can't prove that yet. And it's built for my codebase and my opinions.

To copy it:
1. Find the rule your agent breaks most
2. Put test running behind one script that picks tests, prints counts and fails on zero tests
3. Add a hook that blocks the bad command and prints the fix

Send this to Claude Code 👇
"Write a Claude Code PreToolUse hook that blocks any Bash command piping test output into tail, head or grep. Exit 2 and print: Blocked, piping hides failures. Run ./scripts/check and save full output to logs/test.log"

Every time an agent breaks the same rule twice I stop adding it to CLAUDE.md and write a check instead.

What rule does your agent break most?

[Adil Patel @TheRealAdilP]

@ZeeshhDecodes I'm still experimenting, models still try to find workarounds when running on their own. But it's good start.

[Adil Patel @TheRealAdilP]

@bb833dong Models are great at being positive about everything

[Adil Patel @TheRealAdilP]

@Joshua_WD Deterministic tools around a non deterministic model

[Adil Patel @TheRealAdilP]

Yep that's the next step. Stop and SubagentStop are already wired up but right now they only record. Every edit and every a3q check result goes to a log.

I didn't want the hook running the suite itself. Stop fires every turn, our suites take minutes and parallel agents would queue behind each other.

The plan is the Stop hook reads that log instead. If the agent edited code and has no passing check since its last edit, it returns block with "run a3q check". That's instant and only fires when it should.

Getting a week of numbers first to see if blocking actually moves the 63%

[Adil Patel @TheRealAdilP]

Rough before and after on my own Claude Code logs, not a controlled study. ~9,600 sessions split at the date the hooks went in. Subagents running a test or type check after their last edit went 42% to 63%, and blocked commands didn't count. CLAUDE.md said to verify the whole time and the number only moved when a hook enforced it. Caveats are a small before sample, model and task mix changes, and regex parsed results. Next numbers will come from a3q's own verdicts

[Adil Patel @TheRealAdilP]

@kanekallaway @bot analyze this video

[Gergely Orosz @GergelyOrosz]

I'm starting to sense a divide in the tech industry:

- People thriving on autonomy who are reinventing how they work and what tools they use with AI: LOTS of work, but they're enjoying it and getting very efficient

- People used to being mentored / trained / or simply told what to do and how. Frustrated with the lack of guidance from management on what to do with AI; also frustrated by seeing some of their peers thriving; increasingly bitter about AI and the state of the industry, as a whole

Neither group seems to be aware that there's another group, or have empathy for the other group

[Adil Patel @TheRealAdilP]

This is why I think on paper first. If the agent is in the room while I'm still figuring out the problem, I end up with its idea and think it was mine

[Ethan Mollick @emollick]

Important, but not unexpected, finding that experts working with AI agents are swayed by the positions of the AI, even if they think they are working independently.

If your independent judgement matters on a task, you need to be deliberate about when in the process to use AI.

[Adil Patel @TheRealAdilP]

@DougKennedy93 I think it's more of thinking deeply and understanding the real underlying problem your customers have. To quote Henry Ford. If I asked my customers what they wanted they would say a faster horse.

[Adil Patel @TheRealAdilP]

@DarayuthH Its not x, it's y. Here is what i learned from b2b sales.

[Adil Patel @TheRealAdilP]

@ThePrimeagen Faster CI is a good one

[Adil Patel @TheRealAdilP]

@theo I have a hard time announcing within my own company new (small)features/changes. Its so rapid people having a hard time keeping up. Forget about outside announcements

[Adil Patel @TheRealAdilP]

@rankdim Why I have no friends (manager)

[Adil Patel @TheRealAdilP]

@mhdfaran @Hesamation Also no one can define it

[Adil Patel @TheRealAdilP]

@philhchen How do you prevent interview bias, we are shifting to a vibes based approach. How do you measure hunger, tenacity, and curiosity? How do you compare that between candidates?