Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.
This is true of all agents, not just coding agents. Probably the biggest challenge that most companies run into in their agent strategy is getting agents the right constrained context to work with for a task.
Too much information or conflicting sources, and the agent can easily draw from the data and produce the wrong result. Conflicting sources of truth for documents, data sources that haven’t been kept up to date, knowledge management systems that rely on tribal knowledge to navigate, and so on.
On the other end, of course, too little information and the upside is highly limited of agents in the first place. Thus, a lot of challenges with AI strategies are actually data strategy challenges in disguise.
This is why there’s such a significant premium on getting structured and unstructured data environments setup properly so agents can work with information effectively. Critical for any large enterprise adopting agents, and also a clear benefit in some cases to startups that can be designed this way from scratch.
The most dangerous role in software right now is director or vp who spends more effort lecturing upward and downward on the technology of ai instead of listening to the doers about the real limitations. Propping up huge wins from AI without equal weight to the absolutely unhinged hallucinations that cost an engineer twice as much time.
root issue is most employees aren't in properly motivating environments
it's adversarial where the employer and employee are seeing how much they can get over on the other
it's exactly why ai isn't going to magically make teams more productive
AI slop is good, actually. Slop is what enables fast parallel experimentation. The etiquette and skill is understanding the boundaries of where slop exists and the extent to which it should be cleaned up and how.
A few examples:
I’m working on the internals of some system right now. The API and GUI of this thing is fully zero shame slop. It’s horrible. But it lets me focus on the core quality while shipping a usable piece of alpha quality software to testers (transparent about the slop frontend).
Similarly, this system has plugins. We sent agents in Ralph loops overnight to generate dozens of plugins. The plugins are slop. The quality is bad. The plugin API/SDK is absolutely not done.
But we can test a full GUI with a full plugin ecosystem. When we change the API, we can regenerate them all. The cost of change is just tokens, the velocity is incomparable to before.
I built Terraform. We tested and shipped TF 0.1 with about 3 very weak providers. Because we ran out of time. Building was slow. And when we changed our SDK the cost was immense. Totally different today, 10 years later. Today, I would’ve slop generated 100 providers (again, with transparency and cleanup later, but just to prove it out).
As an anti example, I would not PR this (without prior warning) to another project. I would not throw this onto customers without full review or transparency (as I’m already doing). I would not accept first pass slop. It’s almost never right.
Slop is a tool. And like anything else it’s not blanket bad or good. The context is everything.
Manager: We use agile.
Me: Be honest
Manager: We implemented SCRUM for our tasks.
Me: I said 'honest'.
Manager: We cut Waterfall into sprints.
Me: Thank you
I’m going through the craziest burnout I’ve experienced in my ~17 year career
I’ve been sick for 16 days now, haven’t even been able to go for walks
I kind of fucking hate AI
I think all of these things are related