Warp

Open infrastructure for building cloud software factories

[Warp @warpdotdev]

Introducing Scorers: Agents that grade your agents.

Use LLM-as-a-judge to grade past coding agent sessions on:

• Quality
• Efficiency
• Compliance
• Or any custom dimension

Scores feed into performance measurements and automatic self-improvement for software factories

[Warp @warpdotdev]

Interested in trying Warp Factories? Request early access: https://t.co/5ZLqkJj0wT 🔖

[Warp @warpdotdev]

Three months since adopting software factories, we’re at 40% more PRs merged and 67% lower cost per PR.

We got there by measuring agent quality, benchmarking models, and using self-improvement loops to make our factory better.

Here's how you can do this too in Warp Factories:

[dex @dexhorthy]

Building Evals on Production Data: 🦄 AI That Works #79 https://t.co/tyFxmO60mD

[Ben Holmes @BHolmesDev]

At @aiDotEngineer, I presented our skill called /skill-doctor. It's a skill to improve your agent skills.

It aggregates your past agent sessions, grades their quality, and propose improvements to fix common failures.

[Warp @warpdotdev]

Great introduction to software factories if you've been wondering how they work

[Zach Lloyd @zachlloydtweets]

Here's my take on what a software factory really is

This is my talk from @aiDotEngineer NYC, and I decided to record a version for everyone to watch on X.

tl;dr - software engineering is becoming factory engineering, and there's new work (especially factory benchmarking) that engineers should start focusing on.

[Zach Lloyd @zachlloydtweets]

Getting software factories working is going to be one of the more interesting software engineering challenges in the next few years.

Software engineering is becoming factory engineering. Organizations that can make their factories robust, dependable, and self-improving will be able to ship more at better cost and have a competitive advantage.

[Warp @warpdotdev]

If you're interested in software factories, but you're not sure where to start, this is a good read:

[Zach Lloyd @zachlloydtweets]

Software factories can be a real culture shift. The biggest change for us was moving from individual agents on everyone's machines to one central agent setup (a factory) that we can build as a team.

I wrote some learnings on how to navigate that. https://t.co/9fXUtbO5iL

[Warp @warpdotdev]

Until next time @aiDotEngineer!

[Ben Holmes @BHolmesDev]

We benchmarked some agent orchestration pipelines at @aiDotEngineer. Which would you pick?

[Zach Lloyd @zachlloydtweets]

An interesting engineering challenge this next year will be closing the development loop we've started.

We have small loops already (code review, automatic CI fixes, computer use verification), but we need to close the larger loop: let agents suggest what to build next.

[Zach Lloyd @zachlloydtweets]

Software factories can be a real culture shift. The biggest change for us was moving from individual agents on everyone's machines to one central agent setup (a factory) that we can build as a team.

I wrote some learnings on how to navigate that. https://t.co/9fXUtbO5iL

[Warp @warpdotdev]

If you're building a software factory, this may be your north star:

[Ben Holmes @BHolmesDev]

I've been thinking about what a true, closed-loop software factory will look like.

One where agents know what to fix, what costs to optimize, and the big one: what features to actually build.

We're not there yet. But maybe we're getting close... https://t.co/UJb11hFaEn

[Warp @warpdotdev]

If you want to set up these automations, Warp Factories is a good place to start: https://t.co/5ZLqkJj0wT 🔖

[Warp @warpdotdev]

If you're ready to build a software factory and you're looking for a starting point, here's three automations you can try 🧵

[Warp @warpdotdev]

3. Have agents suggest roadmap items from customer feedback.

For example, you can use @GrainHQ to record transcripts on customer discovery calls, and have coding agents pull out insights.

Here, an agent goes through prior customer interviews with a scheduled automation and looks for commonalities in where customers get stuck. From there, you can fully close the loop by connecting your taskboard via MCP to drive the next agent task.

More on integrations: https://t.co/ToVH2ueBEe 🔖

[Zach Lloyd @zachlloydtweets]

We've found a few key pieces to successful software factories, both from our internal testing and from helping companies deploy their own:

Factories-as-code: you should configure every part of the factory (agent orchestration, automations, permissions, sandbox configuration, etc) as code. This enables testing of different factory configurations to see which are most efficient, highest quality, etc.

Multi-model & multi-harness: you should make sure that your factories are able to use the latest models, both frontier and open-weight, and use different coding agent harnesses like Claude Code and Codex.

Data ownership: you should make sure you store and own all data coming out of your factory. This is the raw material for improving its operations.

[Warp @warpdotdev]

Join @BHolmesDev and @sergical to build a software factory that fixes its own bugs

[Ben Holmes @BHolmesDev]

I'll be joining @sergical at @sentry on October 22nd to talk about connecting error monitors to coding agents!

It's how we've closed the loop in our own software factory to have agents assign themselves work automatically. The flow we're using right now:

- Connect to an alert webhook like Sentry
- When errors happen, start a cloud agent run with the alert and the repo as context
- Have the agent ping humans for code review (or better: with an HTML summary of the problem and the solution)
- Merge and keep monitoring

This closes the loop on reducing software bugs. We'll show you how to deploy it in-code yourself!

[Ben Holmes @BHolmesDev]

Someone asked how I defined a software factory recently, and I think my definition is different from others.

To me, a software factory is centralizing your system of agents into one place to build software. That means going from custom coding setups on everyone's machines to a single definition.

In practice, it's one source of truth for agent automations, repository access, secret access, sandbox environments, model selections... every aspect of the stack.

I think loops and automations are a secondary aspect of this that gets unlocked once you unify everything into an "always-on" agent system. But the heart of it is having the whole team agree on the agent setup, the same way we agree on CI, lint rules, and all the automations we had pre-AI.

This also makes it clear what an engineer's job is now: to make the agents better at building software. Not to be prompt monkeys! We like to call it "factory engineering"

[Zach Lloyd @zachlloydtweets]

I chatted with @hudaman (Sr Engineering Manager, Uber) and @cmhandel (Director of Product, AI Platform, Faire) to compare notes on adopting software factories. We discussed:

- The best starting point for factories
- Code review and spec writing
- How to navigate team culture changes

And how everyone's handling software factory adoption at their companies. Many different approaches to this

https://t.co/sa5lWVrtTq