@geoffreyhinton We need evidence that this is going to be a problem. This alarmism, without evidence to back up the claims, is neither productive nor scientific.
I keep hearing the compiler argument: we don't read assembly anymore, so perhaps soon we won't need to read code either.
But an LLM is not a compiler. A compiler performs a constrained transformation according to well-defined rules. An LLM generates what is probable from what it has seen and the context you give it.
And what has it seen? Brilliant code, average code, legacy code, hacks, tutorials, gigantic codebases, and toy exercises. Most of it cannot be elite software engineering because elite engineering is, by definition, rare. Left unconstrained, AI pulls us toward the average.
That matters because software naturally accumulates entropy. AI has learned from codebases that already contain that entropy and, without strong engineering judgment, can reproduce it in yours at unprecedented speed.
Quality is not the natural resting state of a codebase. Look at curl. Daniel Stenberg and its maintainers have spent decades reviewing, testing, simplifying, and rejecting changes to preserve its coherence. Great codebases don't stay great by accident.
Now look at Bun, now owned by Anthropic, from the other extreme. They use agents aggressively to reproduce issues, implement fixes, write tests, and open PRs. But they haven't removed human judgment from the process. AI can generate the work. Engineers still decide what deserves to enter the codebase.
Use AI aggressively, but don't outsource your understanding. It only takes a few PRs you haven't properly read before you start losing the mental model of your own system.
AI can generate code faster than we have ever been able to write it. The faster AI generates code, the more important it becomes that we understand what it produces, challenge its decisions, and reject code that doesn't belong.
For goodness' sake, read the code.
AI agents are making us question whether mechanisms like PRs are still necessary. If a coding agent can write the code, test it, review it, and fix its own mistakes, why keep the friction?
A PR is not just a code review. It is a governance mechanism. It creates a boundary where work can be inspected, assumptions challenged, quality assessed, and changes accepted or rejected. That boundary matters whether the work was produced by a human or an AI agent.
AI gives us enormous leverage, but leverage weakens direct signal. When one person can produce ten times more work, you inevitably understand less of everything being produced. Remove your governance mechanisms at the same time and you are increasing both output and opacity.
Governance is not about reducing autonomy. Good governance is what makes autonomy possible. Give humans and agents freedom to operate, but never give up your ability to independently assess the quality of their work.
As leverage increases, governance must get stronger, not weaker.
If AI gives you 10x the output, what are you doing to make sure you haven't also created 10x the problems?
Good poets can break the rules of grammar because they have mastered them. What looks like freedom is often built on mastery. Software engineering is no different.
Languages and frameworks constantly give us new abstractions, patterns, and features. Protocols, generics, decorators, metaclasses, hooks, advanced type systems. They can all be useful. But before using one, you should be able to explain what problem it solves, why the simpler approach is not enough, and what complexity you are introducing in return.
AI makes this even more important. LLMs know every obscure feature of your language and will happily introduce it into your codebase. Knowing that a feature exists is not the same as knowing when to use it.
If you can't explain why the simple solution isn't enough, why are you using the complicated one?
I never thought I would have to say this as an educator and CTO: developers are now commenting too much.
Comments are for humans. They should explain the why when the code alone cannot. With AI coding agents, comments have a new risk. A long, confident comment can become a shortcut for an LLM. Instead of exploring the codebase and determining what the system actually does, it may accept the comment as ground truth.
Comments also age differently from code. The implementation changes, but the comment survives. What was once useful context can become a confident description of a system that no longer exists.
That means comments deserve as much scrutiny as the code itself, perhaps more. Code can be tested. Comments cannot. A test can tell you when the implementation is wrong, but nothing automatically tells you when a comment is or has become wrong.
The code is the source of truth.
And if your comment is longer than the code it explains, ask yourself: do I really need the comment, or should I make the code easier to understand?
AI is a tool. You are the driver. Never reverse those roles.
Use AI to move faster, explore alternatives, challenge your thinking, and do things that would have taken you hours before. But speed does not transfer responsibility. If you believe an AI-generated answer is correct, you need to be able to explain why. If your only justification is "Claude said so," you don't understand the problem well enough yet.
The best engineers use AI to amplify their judgment, not replace it. They know when to trust an answer, when to question it, and when to go deeper. AI can help you build something you could not build alone. That does not remove the need to understand what you have built. You may delegate the work. You cannot delegate the responsibility. If it ships under your name, you fully own the outcome.
Using a technology is not the same as understanding it. Experience matters, but what really compounds your value is whether you learn the principles behind what you use and actually apply them.
If you’ve been using a technology for years and still can’t clearly explain its core ideas in an interview or meeting, that tells me something: not about your intelligence, but about your curiosity and desire to learn.
If you use AI without understanding the fundamentals, you won't be able to distinguish right from wrong. You may blindly accept whatever AI sends your way, confidently build on false assumptions, or solve the wrong problem.
In a high-performing team, what you know today matters, but how fast and consistently you learn matters even more. If you haven’t been learning while doing the job, what makes me confident you will suddenly start after joining us?
When the same query costs 20× more on ChatGPT than on Google Search, you realize that the traditional two-sided market, where ads subsidize a free product, simply can't currently support the economics of AI.
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
https://t.co/AUKzoQ5Ikb
Is software going through the same design homogenisation as modern architecture? Different products, different companies, the same visual language. Everything is starting to look like Claude.
Data science is no longer 80% data cleaning.
I’m still amazed by how quickly you can now design and run experiments. What used to take a day now takes an hour, largely because LLMs have eliminated so much of the data-cleaning work.
Between investing on training new LLMs and making existing open models accessible to their citizens, I would prioritize the latter. Europe should not waste resources training new models. It should focus on providing free access to the best available ones.
@claclarke I would be curious to see where the known non-relevant documents ended up in the search results. Given the limited pool depth of the test collection and how capable LLMs are, I wouldn't be surprised if the documents marked as relevant by the LLM are actually relevant.
In SF, the same person can proclaim the imminent arrival of AGI, invest in companies it should render obsolete, build one of those companies, and sell access to the very intelligence that is supposed to dissolve markets altogether.
Every age invents the myth it needs. Ours has chosen AGI.