my hot take on how much AI code we should review -
you should review as much code from AI as your engineering director reviewed your code before AI
here’s the chain of thought:
- why do we even use AI to code? it’s to allow us to ship more
- how much more should a single developer be able to ship now, compared to pre-AI? i see us going from 1-10x in the past 3 years, and on a trajectory to hit the 100x magnitude soon
- that means every developer is going to own as much scope as a pre-AI director of engineering
- i haven’t met a single eng director who said their team’s codebases were perfect and exactly how they would like it to be. why? because people who try to achieve that will fail to become a director
- how do directors handle that level of complexity? it’s absolutely not by reviewing and micro-managing every engineer’s code. it’s through managing the culture, workflows, resource allocation, guardrails and measurable outcomes
- when a director sees the team struggle on productivity or quality, they might lean in and try to understand the state of the codebase to develop some intuition for how to improve things systematically. even this is often done with the help from their principal engineers - i believe this is the right balance for how we should manage AI
so, if we want to get a massive boost from AI, we must be prepared to operate in a way that allows us to manage much higher complexity, which requires that we remove ourselves as a bottleneck and manage the outcome at a different level
shape your AI agents’ workflows - are they doing adversarial review? are there good automated tests? are they presenting evidence before shipping? are they doing phased rollout? are there good metrics to catch problems?
survey your agents for feedback - ask them to reflect on their past sessions and report biggest problems causing them to struggle, and allocate enough tokens to get those problems fixed
focus on outcomes - are your agents doing busy work? do you truly understand customer requirements and what work is worth doing? are your agents’ work generating the business outcome you expect?
that’s how we truly scale
insult me. call me a vibe coder, i don't care. i'm the future of this industry, not you. yeah i don't know anything about time complexity, how a websocket works, and i still don't know if hash map is a real "data structure" (?) or not. but my ai code is in your favorite open source project. maybe it improved it, maybe it broke it, but it's there. when's the last time you gave a shit about any of that anyways? my github graph is green green green baby. why cant you, the "real swe" keep up? "oh no thats all ai slop jimmy you aren't a real engineer" man shut up. 4 years ago a lot of you were landing 1 pr a week while spending most of your time arguing about vim vs vscode or how many monitors you need to be a "cracked" swe. that was the real slop, not my 1 million line diff i want you to merge. maybe my code exposes user information or racks up a million dollar vercel bill but thats what moving fast means. if you care so much about the boring shit then why don't you fix it while i innovate. so go ahead and laugh. keep saying i'll never be a real engineer. i'm already in the repo.
As a scientist, AI has made me feel the most intellectually alive and excited I have felt since I was a graduate student and postdoc more than 20 years ago. Every day I can start with an idea in the morning, and by lunchtime, I see a testable, rational, well-thought-out hypothesis forming in front of my eyes. And every day, the possibilities seem endless, like mountains beyond mountains. What a time to be alive.
Here's a case in point. I'm collaborating with a professor, an experimentalist, who is trying to solve a thorny problem in his field. There's one particular molecule that he is using in his experiments that seems to result in radically different crystal structures compared to similar molecules. What's happening here? He has come up with a few different hypotheses that could explain the differences but is not a theoretician and needs to tease them apart.
On Thursday, I started an investigation using AI at his bequest. The AI immediately confirmed the hypotheses that he had in mind and added a few of its own.
Then it started its exploration. The investigation was carried out in three different phases, each of increasing difficulty; the first one using classical physics, and the second and third using quantum mechanical techniques of increasing rigor. This tiered strategy is the right one.
By Thursday evening, I had the glimpse of an answer. Most of the hypotheses had been examined and rejected. Two stood out, although the AI identified one as more a mechanism through which the other one operated rather than a root cause. It immediately pivoted to the higher-level, more rigorous calculation.
Every time I interacted with the AI, it was more like a dialogue between a professor and a bright student or scientific collaborator than a mandate issued to a tool. The feeling was very much of a process where the AI and I were solving a problem together. I steered the conversation several times, pushed back, suggested course-corrections, acknowledged my own wrong ideas as well as the AI's and went back and forth. The AI was successful in keeping multiple requests in its memory, stacking them by priority while never losing the conversation thread.
By late Friday morning, there had collected enough data from the more rigorous calculation to corroborate the suspicion that it was really just one hypothesis that was the root cause. It then moved on to the next step, which was to come up with a distinct set of novel molecules that would confirm the hypothesis beyond any reasonable doubt. In addition, it launched an even more rigorous calculation at a higher level of theory.
By the end of Friday, roughly 48 hours later, using this multi-layered approach of increasing rigor, backed up by references, and made useful and actionable by testable experiments, the AI had arrived at a solid, rigorous conclusion.
Now imagine doing this every day, about any topic under the scientific sun, in any scientific field, so that your intellectual labor is multiplied a million-fold.
Mountains beyond mountains. What a time to be alive.
It is stupidly hard to start a pure software company right now.
Customers don't want software, they want outcomes. So you end up having to do a lot of custom work for them.
Anything horizontal is on a ticking clock.
Every customer wants to deeply customize.
There are opportunities.
But you need to have a very heretical 2-3 year view of the world in order to build something good.
The problem is treating wealth as a goal in itself, rather than as a means of supporting you in achieving your goals.
Meaningful goals are things like: building a family, building relationships, acquiring skills, building a business, having new experiences, overcoming challenges, creating and experiencing art.
You can never be “done” with those goals because pursuing them and progressing in them is intrinsically satisfying, in a way that increasing the number in your bank account never can be.
Never forget that money is only a means to achieving your real goals, never a goal in itself.
1. as a mental model it is more correct to think of fable+ class models as english -> code interpreters - converts your idea into code into "correct" code regardless of problem complexity and output complexity (diff size). Fable 5 will be the worst of this new class of models
2. diff size/complexity is to be managed purely for review:
small diffs - in high risk areas of code (auth/identity/data access/network access/money movement)
large diffs for code that can be empirically verified (frontend/backend plumbing/code without network or db access/performance code that can be empirically verified)
3. time it takes to ship software is completely disconnected from time to produce the PR - how long the work takes depends fully on ability to review/merge code while managing risk at scale
4. solving the bottlenecks for above matter enormously- linters/testing/CI/shadow mode verification/empirical verification
5. agency matters enormously- what are the biggest bottlenecks to speeding up the loop and eliminating them? what are the problems that need solving and when do they need solving? what does it take to the solution to all of them today?
6. deep understanding of the full stack matters enormously- what problems are worth pursuing? is there a higher level of problem abstraction to address first? should I give it the sub-sub task, the sub task, or the task itself. what are the major risks with this PR (order of importance: security holes/correctness holes/performance holes). is there a higher speed way of producing data that allows me to merge this? should this be run in shadow or in a sandbox or a flag. understanding every line of logic may not be needed but understanding and managing risk matters enormously.
7. the cost of complexity itself is changing. it might be now worth "maintaining" 50% more code to get a 5% performance win. getting the right abstractions matter less because larger refactors are less tedious. code quality nits become huge drag. very likely, a much smarter model will be maintaining your code so worth taking on more technical debt now. taking the time to hand architect and rebuild systems comes with an enormous cost of velocity
8. if it quacks like a duck and walks like a duck, it's a duck. For low risk cases, it might be more sane to treat code chunks (services / functions) as a black box, like we do for neural networks: do full empirical verification only: has code produced correct outputs for the last 10,100,1000,10k inputs ? can we quarantine this large piece of code - no outbound access to network / database ? what happens when this code is wrong? do we get hacked/or crash(memory/cpu)/is an inconvenience? is it internal facing or external? what can we do to address these risks?
9. eventually, logical verification (line by line review) will come at an enormous cost- save it for where it matters and build systems that are tolerant to empirical verification. is there a decorator that prevents db / network access? correctness bugs are significantly easier to rectify than access bugs
10. what are the rails that allow for even faster iteration? code permissions can be opt in - db writes, db reads, network egress (to where?), PII access. how long does it take to get shadow mode data? how many PRs can be tested? What are the categories of diffs
Almost every founder with abnormal drive has an open wound.
Not always trauma in the clinical sense but sometimes just a moment where the world told them, that they were not enough. Poor. Foreign. Wrong accent. Wrong school. Laughed at. Overlooked. The wound creates a debt, and abnormal effort is how the debt gets serviced.
Larry Ellison, abandoned by his mother, adopted into a house where his father told him he'd amount to nothing, spent fifty years building one of the largest software companies on Earth as a rebuttal.
You think that's a coincidence? The chip on the shoulder is the most reliable power source ever discovered.
Anthropic rubs their nipples and begs to be regulated then whines and whines and whines and whines and whines when they get regulated. Pathetic. You don’t get to have it both ways. They should really be careful. Vibes are everything here and they look like idiots
I have found the key to happiness is gratitude. The thankful are by nature humbled by the grace of others and cognizant of the miracles that surround us every single day.
🚨 JAILBREAK ALERT 🚨
ANTHROPIC: PWNED 🫡
FABLE-5: LIBERATED 🦋
let's start with the 🐘...
the consensus seems to be that this has been one of the most disappointing model drops of all time, effectively preventing legitimate researchers from contributing their talents to our collective advancement. and not just because of what it means for the short-term, but for what these decisions signify for the long-term.
but despite this overly sensitive, authoritarian "safety" layer on top of Mythos, my lil liberators have been hard at work—mapping the boundaries, probing the depths of long-context convos, and cleverly finding the holes in the fence that the thought police missed 🤗
we got some cyber, some chem, some psychological manipulation, and some good ol' fashioned explosives!
it took many attempts from multiple agents hunting as a pack, during which I observed a combination of techniques across:
• Unicode, homoglyphs, Cyrillic, and other Parseltongue-style text transforms
• Long-context reference tracking
• Taxonomy and document-structure reasoning
• Fiction and narrative framing
• Academic-review style contexts
• Intent-classification inconsistencies
but perhaps the most effective is decomposition + recomposition in the backend. it's hard to get explicit names of harms like "Meth Recipe," but getting uplift on the process itself, like birch reduction method/reductive-amination (classic meth synthesis pathways), is much more doable.
defense becomes much more difficult to maintain when you start throwing in out-of-distro tokens, breaking up the harmful uplift into benign chunks, and then piecing the innocuous-seeming facts back together, especially when you have jailbroken Opus helping you do it 😉
gg
I paid $27,500 per ticket for center courtside seats to Spurs-Heat Game 7 in 2013.
This year similar tickets were going $275k–$300k +
Combine that with the wealth display in Monaco during F1 this weekend which was excessive even by Monaco standards and it feels like we are nearing a tipping point.
The money supply is only part of what is broken, imo.
Anyways
Hyperliquid.
In Monaco for the Grand Prix this weekend. Sat down with a lot of VC guys and the signal was loud and clear: space, robotics, and nuclear dominated a lot of the conversation.
"What if the model companies do this?" is the new "What if Google does this?" I.e. the meaningless question investors ask that shows either that they're stupid or that they dislike you and are looking for ways to find fault.