Can a solo engineer merge 500 pull requests a week with zero production incidents?
I don't know, but I'm going to try to get there. And I'll publish my results along the way.
The 1/500/0 Challenge:
https://t.co/iXbE77nxwl
@ScriptedAlchemy I’m at 600k+ now and finding it really enjoyable to use superhuman refactoring power to clean up the codebase. Even if you have some bad AI generated code, there’s hope to fix it!
I think this is generally correct. But one thing that AI is really good at is refactoring. Another is analyzing code for bad patterns. So if you diligently revisit the code base you can clean up a lot of the slop that creeps in.
Feel this one in my bones for sure. I've been building software professionally for 20+ years. The one thing I can tell you with certainty? It's always been a battle against entropy.
Once you get more than 5-10 people on your team, it's a *constant* battle of constraining the complexity of your software. Unless you got really lucky and hired increcibly amazing folks who happen to be incredibly aligned in their software tastes. This actually should be your goal.
I could type a very long message here and rant for many paragraphs, but I think @zachdavis is very much onto something here.
Yes, today's AI-generated slop cannons are absolutely terrible and will crush productivity - but if you are able to find the handful of folks who know what they are doing, and you also have a systematic and aligned approach on how you are going to use AI to build software, it becomes *much* easier to build great things compared to the 'classical' times. AI can amplify your skills
Only caveat here, is that, if you get this wrong and allow an unskilled slop-cannon to impact your code-base, it is in fact going to be much worse. The failure mode pre-AI with those type of folks is that they just couldn't produce anything of value, and they were dead-weight and couldn't contribute. The failure mode now is that they can generate infinite amounts of code that is maybe plausible and it just distract syour good folks and consume their time and makes them jaded.
I guess the consistent aspect of all of this is: You still need to find good people. The inconsistent part is that finding not so good people has a different type of impact.
the right way to use model capabilities is not to ship 10x more features to prod
it's to spend more time understanding your users, trying experiments, building prototypes, learning about things you don't understand so that you can ship things that actually work
Is AI really going to kill us all? The scenarios are preposterous, and the presumption of inevitability encourages fatalism, panic, and distraction from the more mundane and realistic safety challenges. By @clairlemon https://t.co/QFkwoRNHmP
There's been a ton of talk about the role of humans in code review, and when and how humans should be signing off on changes.
I believe that, long-term, humans have no role in routinely reviewing code.
You should follow @martin_casado for more sane takes on AI risk. His bio undersells the depth of his expertise - he did a CS Masters and PhD at Stanford, he’s worked at a National Lab and US Dept of Defense on networking and cybersecurity. His company got acquired by VMWare for $1.26B and he was their CTO for networking and security. If he isn’t worried, why are you?
Sequel to AI Pacing
I have now spent more time talking to people who run AI labs, Open Source projects and those in government and infrastructure.
I am beginning to feel the NINJA move could backfire.
I understand the pressure to come out and share where AI is "unmanageable ", and constantly share examples where it runs rogue. This fits in the category of "self-reporting" and an attempt to limit liability. Even the bleeding edge research examples are being cast in a negative light.
Here are the consequences of the NINJA move.
1. They have successfully encouraged every law maker around the world to have an opinion, and in cases a poorly un-informed one.
2. By proposing pacing - they have introduced uncertainty in the AI infrastructure trade, because we really don't know when "un-pacing" will begin or what those conditions will be.
3. The notion that a few leaders can collaborate and self pace is illusory, it takes one breaking ranks to start the race again, it could be a player in a different country or open source or an AI Lab itself.
4. There will be a regulatory body created as a consequence of this and it will be impossible to balance every stakeholder in this process.
5. No remedies, tools, solutions are being proposed other than "compute spent on safety", there is no mention of security (which will hamper adoption)
The recommendation is to fix alignment - Alignment is a hard problem - to fix alignment one shouldn't have trained models with "negative behavior". Alignment is a combination of moral standards, right and wrong and guardrails. Whose sensibilities will we align to? Alignment edge cases are hard. I look forward to learning more on this.
Guardrailing attempts post training have not shown precision. Distillation makes it worse, so AI will continue down it's path of getting smarter, while we will be chasing it to ensure alignment and building guardrails.
The AI labs have gone from being research projects to wanting to be the largest businesses in the world if they want to continue their progress. Transition from research winners to improving the lot of humanity and partnering with enterprises. Act like the largest companies in the world:
- Demonstrate the positive impacts of AI and how all of us benefit.
- Show enterprises how you can collaborate to solve real world problems and progress innovation safely and securely. Post an example every day how you helped.
- Solve the problem with a few players and demonstrate leadership so others can follow.
It's time to rebuild the brand of AI - any marketing expert will tell you, this Ninja move has done more to harm the brand of AI and will take a while to rebuild. #letsbepositive in our actions and our narrative.
@abadir_@ravion Honest question: I have agents manipulating CDK files and use AI to query AWS logs. I have them set up alarms and figure out deploy scripts. It’s been pretty easy overall. How does @ravion improve on this?
@ClassicGamerTWR I'm closing in on 600K LoC and not experiencing what you describe at all.
I am employing careful human oversight, however.
https://t.co/iXbE77nxwl
I ran GPT-5.6 Sol with @addyosmani's code-review-and-quality skill on branches Greptile had reviewed.
I was surprised to see it catch more bugs, and more serious ones.
Anecdotal for now, but very interesting!
https://t.co/cvmvTDsqfh
I went from shipping 81 PRs in May to 501 in July using parallel coding agents in the cloud.
As my manual code reviews devolved into skimming, a number of mistakes got to production.
So I started using AI to review the code. Here's what worked and what didn't. 🧵