After over a year of work we're open-sourcing our AI infra:
It's called Output [dot] ai.
- How do you work with proj. packed with prompts?
- How do you track cost?
- How do you test non-deterministic code?
- How do you create datasets from prod data?
👇
This is why preserving architecture in a system is hard. It requires knowing how the things already there have value and purpose beyond what they currently do.
Yes! binary judges are far more practical for most people, because likert scales (or scores) have too many footguns
All the flashcards are here (inspired by @chrisalbon ‘s flashcards) https://t.co/qfB4WJgX5n
Super excited to be speaking at Replay 2026, @temporalio conference.
I'll be talking about our experience extracting Output[.]ai — our open source framework that helps developers new to AI ship reliable, scalable agents.
#Replay2026#TemporalIO#DurableExecution
One detail from this story: the AI finding credentials laying around. I have moved to scrub all credentials stored anywhere in plaintext on my system. No more .env, no more ~/.aws/credentials, etc. I use fnox with a 1password backend.
Most AI agent frameworks are missing durable execution.
Loops, tools, memory: every framework has these. None of them guarantee your pipeline finishes what it started. When it doesn't, you retry, duplicate a side effect, and pay the token bill twice.
https://t.co/LWxSewzLRw using output and zapier sdk your agent has all the context and information to develop and debug your workflow locally and ship fast to your prod
spent 10 years looking for a way to connect APIs without dealing with OAuth flows for integrations. @zapier just dropped an sdk that finally does it. we built a CRM enrichment wf w/ output+zapier sdk: url→apollo→ai field matching→hubspot upsert. no API keys, no OAuth. code 👇
the "agent harness" is the reason we built https://t.co/DLreTw8sP8 enabling teams to build workflows to production together with code agents. filesystem-first memory, reusable skills, swap any model, no saas fragmentation. everything easy to access to you and your agent.
the human-in-the-loop leg is the practical fix most teams land on. don't cut the exfiltration capability, just queue the action for approval before it fires. built this into an agent console at work, the approve/reject flow is annoying at first but it's the only thing that keeps the blast radius bounded when the agent does something unexpected.
After over a year of work we're open-sourcing our AI infra:
It's called Output [dot] ai.
- How do you work with proj. packed with prompts?
- How do you track cost?
- How do you test non-deterministic code?
- How do you create datasets from prod data?
👇
Some of the agents running on it today: website templates for Lovable from screenshots, connector docs for Airbyte, CVE research and detailed reports.
We finally got to the point where we could finish the extraction and share it with everyone.