"Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies."
This is some scary stuff. Dwarkesh was right to call it agent civilization.
Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad.
Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.
everyone is running into a wall with agentic engineering right now
you don’t hear people talking about it because
1. they profit from selling you “solutions”
2. they want to look smarter than others
so let me be the whistleblower - the wall is called “how does it feel”. agents can’t do it
they can walk right past the ugliest UI or the most obvious bug and don’t say a thing unless that’s what you asked
they can take many screenshots and burn through my tokens but they can’t tell me if my landing page animation looks cool
they can click through my app but they won’t feel the dopamine hit when i physically drag my finger on the touch screen and feel the command dial flow with me in @theSSHHIP
this wall is structural. it’s actually load-bearing 😉
because of this wall, we can’t really throw a ton of agents at any problem that depends on “how does it feel” without constantly stopping for human feedback and steering
if you just let the agents keep running without human input, then your codebase will have more and more things that “don’t feel right”
you can avoid the wall by working on non-human problems, like pure math and a lot of scientific research
but the moment you want to build something useful for humans, this wall is there
if you see people pretend they have automated software factories running thousands of agents working all the time, don’t be afraid. don’t feel FOMO, because the only thing those factories shipped are the factories themselves, with human help
10 things I check in a production incident:
1. Error rate
2. p99 latency
3. CPU
4. Memory
5. DB connections
6. Slow queries
7. Queue lag
8. Cache hit rate
9. Recent deployments
10. Dependency health