@kevinzhuang10@yuwen_lu_ The reason why an agency is better is, at an agency you can have "cross pollination" if that makes sense. When you hire for this role, the expectations will be much higher -- i.e. you'd expect the hire to keep up with the world on their own. Proper agencies deal with it.
@someRandomDev5@mbriggs_dev For things like "I prefer cooler weather in the Summer" we have "measured" as opposed to "proven". "Measured" ties into production which is a whole different story.
I believe we are on the same page. In your trip planning example you would add "I never fly Delta" so that the agent would exclude that from its proposals. You don't care about the how, you only care about the what in this example.
I am talking about the same thing in specs. On a lower level, "never choose Delta" becomes an acceptance criterion on a requirement. Its structured representation allows the agent to code it deterministically, and you can always run a deterministic query to see all ACs related to someRandomGuy5's trip planning.
Peace ☮️
1/ I agree it's not a term. It is a descriptive phrase that indicates the difference between structured agentic "memory" (man, I don't like this neurological terminology either) from unstructured one whose substrate is markdown files. The source of this ambiguity lies, in my extremely humble opinion, in the fact that people humanize agents. But okay, this is semantics. The key thing here is, a structured representation of domain knowledge vs an unstructured one.
2/ You're talking about invariants, if I am reading you correctly, with which I also agree. Your job as a developer is to understand which decisions the agent makes on your behalf, or else all you can say is "I dunno, Claude did it". Better constrained specs as produced by a human+agent team introduce more determinism in what the agent implements.
All-in-all, I believe that we, as engineers, should strive to reduce the non-determinism rather than increase it. Prose does not play well with it, imo.
@someRandomDev5@mbriggs_dev We have taken a different path with my team. We use a structured spec format and tooling around it. I genuinely believe the industry is wrong about using markdown for everything.
I am not sure I agree where this is moving to. What you describe as spec, is prose-driven development if you will. Prose still leaves a lot for the coding agent to decide on its own. More importantly, to reconcile what was done with what was asked, you depend on its reasoning alone.
@zachdavis I have the same thinking. I decided to call what you’re describing a Software Plant. A factory is an orchestration layer and a workflow engine. A plant builds and executes production lines.
There are two components here: product contract (aka specs) and implementation (aka code). In many teams the problem is with both. In my company, we had to spin off a product to deal with these, while keeping a longer term vision in mind.
I will start from the latter. The problem we are solving is extracting org value from the sessions between an individual engineer and an agent. Those include capturing agent questions and human answers, implementation plans and other transient artifacts.
For the former, we adopted a drastically different approach and we don’t use MD as the spec substrate. Prose is for humans. Structured specs are for agents. Doing so has enabled having deterministic evidence during both development and QA.
The third component which is equally as important is signal from production. Chances are, after shipping a release your team looks at the dashboards to judge whether “it’s fine”. This is the next candidate for deterministic automation.
All these three pillars form something I call the Software Plant. A factory is a sophisticated orchestration layer and workflows. A plant is where your production lines live. A production line is a bit more than a workflow.
So introducing measures at different levels enables the team to capture requirements, decisions, plans that represent the product as a living system, and get confirmation that the important stuff stands when in production.
A year ago deciding what should be a part of an MVP was a thing. Today, the timeline is so compressed that an MVP is the entire scope you have on your mind.
#ai
@samongaro_ I call this “the AI trust ladder”. You have to put measures in place to be able to climb it. In the end of the day, that’s how you build up enough personal confidence to ship without reading the code.
@astuyve You guys have to finally stop thinking in tickets. “Ticket this, ticket that” come on. It is a work tracking approach, one among several. Problems arise when somebody starts using it as a substrate for specs.
Interesting take, but I take it even further. There are two versions of the spec, the current and future state. The diff between them is the scope.
Tasks are ephemeral, a derivative of the state diff, and needed only during agent work. They have value, but it has nothing to do with the product you’re building.
The only useful case for a ticket is to track the roadmap. The scope is already in the spec,
More here: https://t.co/40kWCaPOYK
Interesting take, but I take it even further. There are two versions of the spec, the current and future state. The diff between them is the scope.
Tasks are ephemeral, a derivative of the state diff, and needed only during agent work. They have value, but it has nothing to do with the product you’re building.
The only useful case for a ticket is to track the roadmap. The scope is already in the spec,
More here: https://t.co/40kWCaPOYK
“This is a research and product direction, not a claim that CRB has already demonstrated reliable prospective pricing at scale.”
I believe this is Codex writing 🙂 I have seen agents getting too defensive like this even in marketing materials. It’s sorta funny.
On a serious note, you are on a right track. A software factory is an interesting new engineering concept, so it attracts engineers who try to build it. But most of what I have seen are orchestration layers. Not many people understand that a production line is a tad more than a workflow.