End-to-end tests do not work at scale, nor does staging infrastructure. Past a certain number of services, both turn into a liability, and a big one at that. It is nearly impossible to run a "company-in-a-box" environment at scale.
Staging is meant to be a smaller, calmer copy of production. In practice, it becomes its own fragile system. A shared database, shared services, and everything around them all need to be up, on the right version, and in the right state at the same time.
Running tests across services on top of that staging infrastructure is even more fragile. Test for service A now depends on service B being deployed, service C having the right data, and the staging database not being mid-migration because of someone else's change.
One flaky service and the whole suite goes red, for a reason that has nothing to do with the code being tested.
When that happens, nobody owns the failure, and it somehow becomes the responsibility of a central team (most likely DevOps, SRE, or Platform. Yesss... been in the same situation a decade ago).
What works instead is each service owning its own tests and mocking the responses of the services it depends on. No shared staging database. No waiting on five other teams to deploy before you can run a single test suite.
This gives you a clear separation of concerns. A service is responsible for its own contract and its own correctness, nothing more. Cross-service behavior gets verified through contract tests between two teams, not through a shared environment that everyone touches and no one owns.
It also shifts accountability to the left because there is no one to blame. If a mock does not match reality, that is a contract problem between two teams, caught early by the team that owns the code.
Fewer moving parts, less shared infrastructure to babysit.
Somehow, every company struggles with, there is no end to this problem, and there is probably a billion-dollar business to be built solving it :)
Hope this helps.
Something I have been thinking about: in the past, the best engineers I knew spent a lot of time automating their work in various ways. Better vim/emacs automations, writing lint rules to catch repeat code issues, building up a suite of e2e tests so they don't need to smoke test the app manually. These kinds of things were the highest leverage activities an engineer could do, because it multiplied their own output, which in turn meant they could build more things.
I think many of these automations have become even more important now. This is true for a number of reasons.
First, infra and DevX automation speeds you up. And if you are running an army of agents, each of those agents will be sped up also. More automation == more output per unit of time.
Second, moving things to code improves efficiency. Your agent could fix an issue every time it sees that issue happen, but that uses tokens and might miss cases. If Claude instead writes a lint rule, CI step, or routine, that class of issue can be fully automated forever. This is really what people are talking about when they talk about loops -- it's about automating entire types of busywork rather than solving them one off. This isn't a new idea at all. Engineers have been doing this for a long time!
Third and most importantly, automation makes it possible for others to contribute to the codebase more easily. Increasingly what I am seeing is engineers are contributing to codebases on day one because Claude can navigate the codebase for them, and that non-engineers are able to contribute to a codebase as effectively as engineers can. What gets in the way of both of these is domain knowledge that lives in peoples' heads rather than in automation -- the stuff you used to have to learn when ramping up. What has changed thanks to agents is the domain knowledge that can be encoded as infrastructure is no longer limited to what is expressible in lint rules and types and tests; it can now capture nearly all domain knowledge, encoded as code comments and skills and CLAUDE.md rules and memories. If I put up a PR for an iOS codebase I don't know and a code reviewer rejects it because it doesn't use the right framework, or if a designer builds a new feature and it gets rejected because it doesn't follow the right architectural patterns, these are failures of automation.
Every team should be writing the CLAUDE.md's, REVIEW.md's, skills, and docs that enable agents to productively work in their codebase with zero additional context from the prompter. This sounds crazy, and at the same time is a natural extension of the stuff engineers have always done: automate, and encode domain knowledge as infrastructure. As the model gets smarter and as the harness matures, this task becomes easier. In the meantime, it is on every team to look for ways to convert their domain knowledge to infra so that Claude can write code better, so that code review catches issues automatically, and so the next person working on your codebase can contribute more easily.
Hi, this is an experiment we launched in March that was meant to prevent account abuse from unauthorized resellers and protect against distillation.
The team has landed stronger mitigations since then and we’ve actually been meaning to take this down for a while. We merged the PR and this should be fully rolled back in tomorrow’s release.