The funniest part of the AI debate is watching career politicians discover frontier AI five minutes ago and immediately decide humanity needs their guidance.
AI getting good enough to perfectly fake anything is not bad, in fact it may be the best possible thing:
Once everyone realizes everything online is fake, the only thing to do is go back to those things that cannot be faked.
Well, either that or melt down into nothing, either is fine.
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Endorsed. The kind of totalitarian control that doomers and decelerationists want is a far more certain danger to our future than runaway AI. I would much rather risk the latter.
This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion. Let me show you how it works:
1.) This guy, with minimal followers and no previous account activity, goes to the Wall Street Journal which publishes an exclusive with quotes from him on his resignation 18 minutes BEFORE this post goes up. Planning was clearly done in advance.
2.) Within hours, it has tens of thousands of reposts and the account has 100k+ followers. The post is punchy, quotable, it almost seems professionally written. The first three accounts to quote tweet it all do so within 15 minutes of the initial posting. Remember, this account had basically zero engagement beforehand, so an organic reach explanation seems unlikely.
According to Grok those accounts are @_NathanCalvin (General Counsel at Encode AI), @peterwildeford (Head of Policy at the AI Policy Network), and @DKokotajlo (Head of the AI Futures Project), all of which are up-and-coming AI-Doomer policy advocacy nonprofits.
The AI Futures Project website says it is funded “primarily” by the Survival and Flourishing Fund, which says on its own website that it has advised Jaan Tallinn, Skype creator and one of the leading investors in Anthropic, to grant over $2.5 million to the AI Futures Project since 2024.
Encode AI says on its website that it is ALSO funded by the Survival and Flourishing Fund, which in turn says that it told Anthropic investor Jaan Tallinn to grant $516,000 to Encode AI in 2025.
And wouldn’t you know it, the Survival and Flourishing Fund ALSO says it told Jaan Tallinn to grant $2 million to the AI Policy Institute, the 501(c)(3) affiliate of the AI Policy Network, as well.
What are the odds that the first three quote tweets of Coxon’s post would all be major AI-restriction policy advocates funded generously by the same donor, who also happens to be one of the leading investors in, and a board member of, Anthropic, the company Coxon was resigning from? And all within 15 minutes of posting (two within ten)?
3.) Jacob Coxon doesn’t have much of a resume, but we do know that, in 2022, he got a $20,159 scholarship for the “long term future scholarship program” from the Good Ventures Foundation, one of the philanthropic vehicles of Dustin Moskovitz, a notorious AI-doomer who has spent tens if not hundreds of millions on policy advocacy to strictly regulate AI, while also being an Anthropic Investor himself.
It also just so happens that the 14th person to quote Coxon’s post was @MaxNadeau_ (27 minutes after posting) who is the program officer for the Technical AI Safety team at Coefficient Giving, another of Moskovitz’s philanthropic spending vehicles. Max is not a frequent poster, his last posts before quoting Coxon were before Labor Day, but he was remarkably quick off the mark for this one.
4.) Basically every major Democrat politician and candidate has suddenly glommed on to this post, and conveniently, as the people cry out foe answers, Bernie Sanders already has a bill written to “ban super intelligence” and regulate AI into oblivion, and will be releasing later this week. The bill, among many other things, will create “a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence” that will be “advised by an Artificial Intelligence Advisory Board comprised of experts on artificial intelligence.” Do you think, perhaps, Anthropic and its many investors who fund AI policy advocacy might have interest in getting to place a pet “expert” on the board of an entity that dictates what AI is and isn’t allowed to do? And isn’t it fortuitous that this whistleblower came forward with his oh-so scary stories so close in proximity to the release of the most radical piece of AI legislation ever introduced?
The board of directors panicked after reading an article in Forbes and hired a boutique cybersecurity firm.
They paid $80K for a team of external auditors to stress-test our network architecture.
A guy named Connor showed up to my office with a customized laptop covered in hacking stickers.
He ran a preliminary port scan and immediately found 14 open vulnerabilities on our primary subnet.
He barged in and demanded to know why our main firewall was running outdated firmware from 2014.
I leaned back in my Herman Miller Aeron chair and smiled like a disappointed father.
I told him he had successfully discovered the outer layer of my decentralized honeypot matrix.
I explained that we intentionally leave legacy ports open to trap malicious actors in an infinite digital loop.
I used the phrase "asymmetrical threat inversion protocol."
He stopped typing on his mechanical keyboard and looked deeply confused.
I told him that patching those obvious vulnerabilities would immediately signal to international syndicates that we were a high-value target.
I said that by maintaining the illusion of baseline incompetence, we achieve absolute digital camouflage.
I asked him if his boutique firm didn't understand the fundamental concept of psychological network warfare.
He started aggressively backpedaling and nervously closing terminal windows on his screen.
He said his scan was just a preliminary assessment and he didn't want to disrupt my advanced architecture.
I generously offered to sign off on his security audit if he classified our vulnerabilities as strategic perimeter decoys.
He rewrote his 40-page compliance report to praise our innovative approach to threat management.
The CEO called me into his office the next day to personally thank me for outsmarting the hackers.
I didn't tell him the firewall was unpatched because I forgot the admin password 6 years ago.
I haven't logged into the security console since the Obama administration.
I billed the company $400 for a catered lunch with Connor.
I spent the afternoon watching a documentary about deep-sea fishing while the company felt safer than ever.
True security is just aggressive confidence in the face of absolute negligence.
For reasons that have remained a mystery to, I was once invited to a dinner conversation at the Royal Society in London. That was some time before COVID, I think in 2019. Also present, believe it not, @demishassabis and @wtgowers and some other Nobel Prize winners and important people.
Not saying this for the name dropping though I guess it's slightly amusing, but to explain where much of my opinion about AI originated from. I really only went because they paid the flight and I rarely turn down a free trip to London.
(I also, years ago, talked to someone from a company called OpenAI who went on and on about AI safety which I thought was all very boring. But I digress.)
In any case, one thing that stuck to my mind is that Tim Gowers had clearly spent a long time thinking about what AI could and could not do in maths, and what "creativity" in proof-leading even means, and whether it can be automated.
His response, if I recall correctly, was basically that the evolution of proofs is a sort of meta-extrapolation in method. Creativity doesn't come out of nowhere, and the human mind is not unique in being able to inject a certain random element.
We also see this, of course, in physics, where "new" ideas are often obviously generated following one or another template. An extremely common template in the past decades is for example to combine two earlier ideas. The problem is that this generally makes the hypothesis even more contrived. It is imo a strategy that should be abandoned.
Physicists are also well-known for literally sneaking up on maths seminars and then asking "what could I do with this piece of maths" (or in the case of Veneziano, looking up pretty integrals in a table).
My point is that (a) clearly mathematicians have been thinking about what to do with AI long before ChatGPT hit mainstream, it's not like they're all super surprised by the current events and (b) there is no evidence that new ideas come magically out of nowhere. They're usually connecting dots that the mind -- human or artificial -- has been presented, in one or the other way.
I don't think there's anything in the idea-generating process that a computer cannot, at least in principle, reproduce. It is possible of course that AI will forever miss some aspect of human cognition, but I consider it to be extremely unlikely.
Whether LLMs will be able to get there is another question entirely. As I have said many times previously, I think the answer is 'no' because (as I saw @elonmusk recently also pointed out), language is a poor representation of reality.
Think about it: It's a tool that humans have developed to transfer bits of information from one human brain to another. Human cognition is already a faulty representation of reality, language is even more faulty, and transferring it brings in even more mistakes.
Language does contain some relations about reality correctly, but the idea that one can reconstruct an understanding of, and the ability to interact with, the real world from language alone seems insane to me. It's having it entirely backwards. You want to start with the foundation of reality instead which is, ultimately, physics. Hence, what they now call world models is in my opinion exactly the right path to go.
The reason LLMs work quite well for maths (and coding), I think, is that in both cases the language is extremely exact itself and it is entirely self-referential, plus LLMs can be beefed up with neurosymbolic software.
In physics, you have the additional complication that not only do you need mathematical structures, these structures must be faithful representations OF SOMETHING in the real world that you must understand in the first place.
It's not as easy as just saying "fit this data". Because that opens an entire rabbit hole of having to understand the data and the experiments and their relevance and how seriously to take it and what it means to fit the data well and so on, to all of which, I am afraid to say, there is no one clear answer. It's all very tacit knowledge that is to the most part not contained in any public record.
This is also why I say we will almost certainly see an AI slop wave in the already shittiest corners of theoretical physics, exactly because the present AIs are not yet good at actually coming up with theories relevant to the real world. You can however totally use them to create yet another idiotic paper about a non-existent dark matter particle. And since this bullshit unfortunately is still getting published in journals, there will be many, many of those going forward.
The only other thing I remember from that dinner is some sort of pink beetroot jelly that vaguely tasted of vinegar. English food isn't for everyone.
Many people think any given ML project is 99% training.
In reality, it’s 50% evaluation, 40% data cleaning, 8% integration, and 2% training.
The first two set the noise floor for learning. No ML magic matters; the model cannot lower the noise floor, as that’s the optimal bound of Shannon encoding of your data.
Thus, not a single day goes by without me thinking about ontology. Even the old labels have to be constantly reviewed.
USA. Summer. It is 95 degrees outside, and I am shivering inside a sandwich shop.
I have discovered how Americans forge strong souls.
Outside, the sun is trying to kill everyone. Inside this small restaurant, it is winter. My breath does not fog, but it is thinking about it. A man near me is eating a cold sandwich while wearing a jacket. In summer. Indoors.
In Japan we would simply turn it down. Americans do not turn it down. And now I understand them better than they understand themselves.
This cold is not an accident. This cold is a gift.
The owner has built, inside his shop, a second season. He invites you in from the brutal heat and hands you the one thing the sun has denied you all day: a reason to be cold. To endure it is to be tempered. You walk in soft and sweating. You walk out sharp and clear, a slightly stronger person than you were.
So I did not complain. I removed my outer layer and offered it to the woman at the next table, who was hugging herself. She said, "Oh, no, I'm fine, thank you." She was not fine. Her lips were blue. But she, too, understood the training. She would not break first. I respected her deeply.
The owner asked if everything was okay.
"It is perfect," I said, through my teeth, which were chattering. "Thank you for the winter."
He said, "...I can turn the AC down if you want?"
I told him no. A man does not ask the mountain to be shorter.
I stayed two hours. I ordered a hot coffee to survive. Then a second one, to hold. By the end I could no longer feel my hands, but my spirit had never been clearer.
So now, on the hottest days, I seek out the coldest rooms. I sit. I shiver. I sharpen.
And when I finally step back out into the summer heat, and it wraps around me like a warm bath, I feel it.
Reborn.
A man who has survived the winter, in August, indoors, for the price of a sandwich.
I didn't. I read every paper as a PhD. And then my advisors got angry at me when I said I couldn't figure out how to ground certain claims in the literature, because the literature was contaminated.
And yes, you should be banned. Not "I guess". 100% banned starting today. Imagine how much taxpayer money you have wasted over the course of your career.
It’s very hard to reproduce the experience of getting a PhD. It’s not just having the space and explicit support to do multi-month deep dives in an adjacent areas that spur new ideas (coursework), or deeply internalize content by teaching it (TA work), or participate in multiple reading groups per week with other highly motivated researchers with fresh eyes (your PhD cohort), or have a senior researcher who has self-selected into making a career out of mentoring apprentices (your advisor), though it is all that. It’s also just the separation from commercial pressure that is squeezing most AI work today into the regime of what could be relevant in the next 6-12 months instead of more blue sky bets on what will be piquing in relevance in the next 3-5 years.
Please please please. I'm on my knees begging every AI exec on the planet. Just stop with this stuff. Stop.
Just give us models. Let the collective distributed intelligence of people figure things out in real time like we always do. Let people adapt. It's what we do.
It's all just so tiresome. We just want models. We'll figure it out. We promise. We don't need societal level surgery and UBI and robot taxes and ham-fisted legislation and populists politicians passing dumb law after dumb law and lobbying groups and all this craziness.
We are not giving birth to magic super miracle machines that suddenly invalidate every single pattern of the entirety of human history and technological development.
We're not.
Really.
AI is amazing. It's wonderful. But it's not magic. Can we please just let AI be cool and useful and problematic in realistic ways instead of all this crazy talk?
We are hallucinating at a collective scale. It's a madness really. A societal meme level madness.
Just give us a products please and leave all the politics in the garage. Stop proposing societal level surgery with drastic measures for things that have not happened and may not happen and probably won't happen.
Just stop.
with tinygrad, the exabox red will function as a single very large GPU. 720 dies x 128 CUs x 64 threads, and you can dispatch kernels to them all at once. want to distribute a Tensor across 26 TB of RAM?
the EPYC CPUs will be glorified PCIe switches. they will network boot into a minified Linux with 0 state persisted, just like how little GPUs boot.
the problems of scheduling, pipelining, communication, synchronization, memory allocation, and locality are the exact same at every level. unlike other libraries and stacks, tinygrad will make you feel like they are. train a 10T MoE model as easily as you train MNIST.
@distributionat ugh there is no AGI. there is no magic threshold. you guys see autoresearch change the random seed from 42 to 137 and OMG ITS AGI ITS OVER yet you critique the junior engineer for the same crap. the cost of dev is falling. overpaid eng struggles to compete. that's the story.
Tips for increasing IQ, based on the latest science:
- If you’re planning to get a PhD in comms, switch to astrophysics (causes much higher IQ)
- If you like working with your hands, become a neurosurgeon instead of plumber
- Get a higher-paying job (lots of data on higher earners consistently have higher IQ)
- If you’re really optimizing for IQ, consider winning a Nobel Prize.