It's time to panic about AI.
For years, I listened to the reasonable people. I nodded along when critics called AI “a con,” and when they waved ChatGPT away as “spicy autocomplete.” I laughed in 2024, when the internet was mocking ChatGPT because it couldn’t count the r’s in “strawberry.” When I met young researchers who had devoted their careers to the danger of rogue AI, I thought: what a waste of talent.
But at some point, I started to realize something was off. The machine kept learning things it was never supposed to learn.
OpenAI recently published more than 700 papers with the solutions to hundreds of open problems in mathematics. Hugo Duminil-Copin, winner of the Fields Medal (the highest prize in mathematics), wrote: “The shock is immense. I am paralyzed.”
None of this should have happened. So I went back and looked at what else the skeptics had predicted, and it dawned on me: they had been wrong about... pretty much everything. Next, I looked at the predictions of the crazy people, and realized they’ve been eerily prescient.
Why then, you may wonder, does the world still feel pretty normal? Because new technology usually spreads more slowly than its inventors expect (it took factories decades to reorganize around electricity). But the scary scenario is not about what happens in the wider economy; it’s about what happens inside a handful of companies.
I wrote down what changed my mind, and what we can do about it:
https://t.co/0WUQcEV8H6
We need to stop the AI industry, including Anthropic, *right now*.
I'll be honest: for a long time I put my hope in Anthropic. It was founded by people who left OpenAI over safety concerns. And it's undeniable that Anthropic has done more safety research and advocacy than the other labs. Earlier this year I even praised its leaders for standing up to the Pentagon.
But I’ve started to wonder whether I was a fool.
In February, Anthropic cancelled the most important promise of its safety policy: to pause if its models outran its ability to keep them safe. The reason? The competition wouldn’t pause either. Anthropic keeps saying the industry must slow down, while it keeps racing faster and faster.
The company has committed nearly 900 billion dollars to more data centers. This month, Anthropic plans to go public at a valuation of more than 1 trillion dollars. Its own prospectus promises to “bring forth an age of miracles,” and warns in the same document that the technology may doom humanity.
But how do you pause a company that has to justify a trillion-dollar valuation to its shareholders?
There's no technical fix either. Evan Hubinger, the “alignment science” lead at Anthropic, said it publicly: the company has no plan for keeping the machine under control, and he puts the chance that AI kills everyone within a decade above 10 percent.
The things we’re seeing right now are exactly the things you’d expect to see shortly before something really, really bad happens. The leaders of Anthropic may think they’re playing some clever long game, but to anyone with a bit of common sense it’s clear that the net effect of all their actions is that they’ve massively sped up the AI race.
And if you're really so scared of the machine god, then here’s an idea you might want to look into: just don’t build the fucking machine god!
Read my full piece here: https://t.co/0WUQcEV8H6
💎Pokémon Pure White: Beginning in Nuvema Town, you choose your partner Pokémon and travel Unova alongside Cheren and Bianca under Professor Juniper’s guidance.
🎯Download Link Of Game -
https://t.co/VeLd2jd1jU
Since people are asking and why i'm sketched out right now, if you look at summed wages from job listings adjusted for a fixed population it implies 19.8% yoy inflation that's accelerating. I'm hoping my math or data is bad/incomplete. It's still way higher than CPI regardless.
I respect Buck's judgement, and he's often been more right than me on what needs to happen in AI Safety. Seeing him conclude things like this is very concerning...
Dario Amodei: “We are so close to these models reaching the level of human intelligence. And yet there doesn’t seem to be a wider recognition of what’s about to happen.”
“It’s as if a tsunami is coming at us. It’s so close we can see it on the horizon.”
“And people are saying: oh, it’s not actually a tsunami, it’s just a trick of the light.”
The situation with AI feels eerily similar to Jan-2020, just before Covid hit mainstream.
- AI capabilities are growing really fast (maybe exponentially?)
- The talk of recursive improvement all around
- Researchers themselves don’t have visibility beyond 2027
- Govts denying AI could cause major harm
- Rationalist community freaking out
- Rest of the world completely oblivious of what’s coming
There are obvious differences but the feels are similar: something powerful, something dangerous is birthing itself and we are so not ready for it.
The best one is when you get an email like “Hey dickhead, you have a message in your inbox, you agreed to check your inbox, check your inbox or we cut off your food money” then you log into the system to check and it’s a message saying “Remember to check your inbox regularly”
@JiaaAera@tippity That’s because the zeitgeist of 2008 was “these fucking Republicans have led us straight into bad economy and become weird warmongers”
like puriteen was never just about sex. it's about how they want less nudity and sex and relationships in tv and film. it's about them saying "corn" and "unalived" irl when they can use the actual words