anthony

agent harnesses // tool-calling abstractions // runtime loops

forcing text models to act like real software

[anthony @0xagility]

demo of the agent orchestrator Ive been building for the last couple of weeks. It's node based so you drop agents on a canvas, wire them together, and each one gets its own model, prompt, tools, skills, sub agents and cost limits.

my theory is that some agent harnesses fail by trying to cram everything into one all purpose prompt. a "senior engineer" persona that researches, implements, tests and reviews in a single context window does all of those things worse than a chain of narrowly specialized small context window agents with shared state memory.

so instead a software workflow here might be seven nodes (understand -> research -> design -> implement -> test -> review -> deploy) augmented with routers, checkpoints and logic gates with each node permission restricted so no part of the processes has more access than it needed for the job.

lastly, loop zones work by allowing any agent to send its output back to any earlier node(s). instead of failing a run entirely the reviewer agent might directly critique the implementer, who then picks the work back up with warm context instead of starting fresh.

this demo is the simplest possible run. one agent, a manual trigger, some tools, and a fork.

[anthony @0xagility]

@jross7878 .com or .ai ideally

[anthony @0xagility]

lost an auction for agentically(.)com for $8k in march and now of course I actually need a solid agent related domain.

cant believe the serial domain buyers were right all along. what a sick joke this is.

[anthony @0xagility]

@tamas_szuromi @LangChain you'd choose which objective you want to improve on out of reliability, quality, cost, and speed. optionally add a set of rules in the brief to really specify what you're looking to improve and how.

[anthony @0xagility]

WIP: recursive self improvement but for agent workflows. the pipeline reads it previous executions and traces and rewrites itself based on your configuration and objectives.

open sourcing soon. @LangChain calling all graph nerds

[anthony @0xagility]

@zeeg the future is now

[anthony @0xagility]

@Arcturus_f @LangChain hadnt heard of it until now but @FUCORY def earned the follow. awesome looking project

[anthony @0xagility]

@mattpocockuk "my snack agent just ordered 18 cheese burgers and blamed it on the diet agent"

[anthony @0xagility]

@WillNessAI thanks. its a nice UX trick to make a slow build process feel that much quicker.

[anthony @0xagility]

how many of the ai startups you saw pitched this year are essentially selling a single workflow in this app?

this is my builder agent creating a multi agent workflow from a few paragraphs. nodes, tools, shared state, review loop, done.

[anthony @0xagility]

@oh_no_sergei not a stupid idea. the nudge already does similar with haiku 4.5 and catches the context specific cases pretty well. still not sure it beats just letting the smarter main model notice the skill inside its own context though. only more testing will tell

[anthony @0xagility]

building a node based agent orchestrator where every agent node gets its own skills, tools and sub agents and I wildly underestimated skills. the wiring was pretty easy but getting the agents to actually use skills reliably has eaten more of my time than any other feature so Im summoning the skills wizard himself @mattpocockuk for a bit of a sanity check.

after a lot of runs I keep seeing two different failure modes. the first being invisible, where the agent should activate a specific skill at a specific step and just never realizes it applies. there's no error or way to trace and the run just continues on until you realize at the end that the output is worse than it should be. partly patched this with nudges, where the harness looks for keywords and concepts and drops a hint like "there's a skill that might fit here" which helped but feels like treating the symptom.

the second is the skill activating and failing anyways. sometimes it's outdated or wrong, sometimes it assumes tools that node doesnt have access to and sometimes the skill is fine in principle but the agent still fails to properly execute the steps, which I think is mostly a context problem. I also suspect my nudges are quietly creating a third issue, where skills fire when they shouldn't

does this match your experience with skills, and how do you actually alleviate each issue? the invisible one especially since its the only one I cant instrument my way out of.

[anthony @0xagility]

@timmoshu currently computer use only. headless (API /structured gen) + MCP are both on the backlog. so you'd be able to spin up workflows programmatically at scale, drop straight into a CI/CD or backend services and hopefully cheaper and more deterministic.

[anthony @0xagility]

@jamesvanderhaak open source soon. ideally a week or two.

[anthony @0xagility]

@jerryjliu0 whoever nails an apple level ecosystem lock in revolving around ai tooling and intelligence will be the richest company on earth

[anthony @0xagility]

from a few hundred runs it does seem intelligence plays a significant role. opus 5 / fable were noticeably better at both detecting when a skill should be applied and actually executing it than older ones like opus 4.8/4.6. similar pattern shown with sol 5.6 vs earlier models.

skills also worked way more reliably when they were fully pinned in the agents prompt vs just indexed with a short description the way tools usually are. the exact wording of the skill (or short description if you do index them) has a huge effect on the call rate so you can essentially fine tune how often the agent reaches for a skill just by how urgent or broad you make the language, but its extremely meticulous

I dont think itll ever be deterministic, but Im hoping with the proper harness configuration it can be improved. appreciate the support, more to come!

[anthony @0xagility]

@dannyzlo appreciate it. I use it for things here and there but definitely not a daily driver just yet. skills, memory, custom tools, and a few other supplementary nodes are required before I can create the workflow diagrams I'm looking for.

[anthony @0xagility]

demo of the agent orchestrator Ive been building for the last couple of weeks. It's node based so you drop agents on a canvas, wire them together, and each one gets its own model, prompt, tools, skills, sub agents and cost limits.

my theory is that some agent harnesses fail by trying to cram everything into one all purpose prompt. a "senior engineer" persona that researches, implements, tests and reviews in a single context window does all of those things worse than a chain of narrowly specialized small context window agents with shared state memory.

so instead a software workflow here might be seven nodes (understand -> research -> design -> implement -> test -> review -> deploy) augmented with routers, checkpoints and logic gates with each node permission restricted so no part of the processes has more access than it needed for the job.

lastly, loop zones work by allowing any agent to send its output back to any earlier node(s). instead of failing a run entirely the reviewer agent might directly critique the implementer, who then picks the work back up with warm context instead of starting fresh.

this demo is the simplest possible run. one agent, a manual trigger, some tools, and a fork.

[anthony @0xagility]

@KevinLeib22 thank you! likely open sourced in the coming weeks once it has a bit more polish.

[anthony @0xagility]

@vasuman Im just glad I no longer need to threaten to off myself in order to get gpt 4o to return the json schema I asked for

[anthony @0xagility]

@yoheinakajima "connect to my bank account and make the overdrafts vanish, make no mistakes"

[anthony @0xagility]

absolutely agree! a few different types of memory are actually in the backlog currently. one being the per pipeline state machine where you can opt in specific nodes and grant them the ability to modify a shared object as the workflow progresses.

the other one is more of an episodic workflow memory, where the system becomes self aware of its own executions, automatically storing successful patterns, failure modes and useful intermediate states so future runs can retrieve and build on them.