I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
Our guidelines for submitting AI-generated code are now up in our repository!
As for all the AI bros seething on our socials, we're simply blocking you.
Learn how to debug, code, and leave behind something useful to humanity when you're gone, instead of peddling slop.
Human oversight is fundamentally necessary for producing anything consumed by humans. In fact, anything consumed by an entity with a value system must inherently be produced by an entity *with a similar value system* (this is why trade can extend across some national/cultural boundaries but not all). Obviously some value structures are more relevant in some domains than others—trading bare necessities (energy, food, shelter) is easier than complex value-related products.
“Intelligence” sufficient to replace human work makes this doubly true, because specific value structures (e.g. creating goods that are not to the detriment of their consumer) are not intrinsic to intelligence nor can they be explicitly programmed.
Furthermore, given that software is (for all intents and purposes) an ~infinite space, human demand can expand into that space arbitrarily, requiring more and more software work. Every serious programmer knows the feeling that there is effectively an unbounded amount of work to do, and projects to investigate, almost certainly until they get bored or die. Thus, even with improved efficiency and fewer humans needed to do oversight on any one project, you’d still expect a larger number of human overseers in the end.
In practice, given that the so-called “intelligence” is extremely limited, it’s capable of producing a small fraction of that total space. Thus, you’d expect supply for that fraction to rapidly increase, despite a fixed limit on demand for it—demand will be magnified in areas with much less supply, which require greater human oversight and authorship.
Nobody tells you this: You can get pretty damn far in life by just being someone people can count on to show up and do the work. Reliability is the cheat code. Stop overcomplicating success. Show up, do the work, repeat. That’s it.
Software engineers don't get paid to write code; they get paid to solve problems.
The faster you realize this, the sooner you'll stop being afraid that AI will replace you and the better your career will be.
Our willingness to read is the most powerful and limited counter to slop
People don't want to read the manual and resort to havingAI code for them
People don't want to read code and just approve it
I bet you anyone who reads O'reilly books produce a TON less slop
I keep thinking about this fix. The code on the left was in production for at least half a year, and technically, it was doing the right thing. Until someone created a large enough volume, and the process started being OOM-killed in a loop, causing an incident. Software development is so much more than writing code.