I feel like I'm taking crazy pills with how many smart people I read and respect are saying stuff like this.
The idea that the AI labs are suddenly, only now, just in September of 2026, advocating for AI safety, and that they just came around to this position bc of sudden financial precarity, is just completely, utterly, conclusively, 100% wrong.
Here's Dario Amodei telling Ross Douthat in February that he agrees with the case for slowing down; that he's in favor of "collaborating" internationally to organize a slowdown; that "I would be all for" a global slowdown. This was 7 months ago, when Anthropic's annualized recurring revenue was rising faster than any company in modern history. He's been saying stuff like this for years, when Anthropic was worth millions of dollars and when Anthropic was projected to IPO for trillions.
[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]
Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:
1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.
2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.
3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.
4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.
5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): https://t.co/TZOm3LfptY
I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
@suekhim Have two kiddos, 10 & 12. The 10 year old definitely needs help with math. The 12 year old would probably enjoy exploring new subjects (he's more advanced) Would love an invite! (Is this tablet or laptop based?)
@OfficialLoganK@theo But it's about where you place it on the frontier. Now it's less intelligent but more expensive than the competition (5.5 low/med). So this only wins where there's a need for speed. There's a place for that, it's just much more limited than the flash models of the past.
@exploreabitmore@stevemagness Heart conditions might experience a U pattern, but overall health continues to improve with more exercise. Can't look at it in a vacuum. Increased afib etc. is offset by reduced cancer and other factors that continue to improve with more exercise
@anshnanda@thesagardevkate Are you using the official API or third party inference / open router? My experience is that third party inference often takes a few weeks to work out the kinks and the main way I notice that is tool call failing.
@drvolts Interesting. So does it have three coils? One outdoor, one indoor, and one water? And then it has a 3-way reversing valve between moving refrigerant from (outdoor to indoor) to (outdoor to water) to (water to indoor) etc.?
Where do they see rates that make this attractive?
Really bizarre that *mainstream* world famous scientists are essentially saying we won’t survive the next 80 years on the course we are on, and most people - including journalists and politicians - aren’t interested and refuse to pay attention. Video by @TTTMediaXR
The Senate proposal aimed at accelerating the adoption of #heatpumps would save consumers money and boost domestic manufacturing, our guest authors @energysmartwv and @GardMurray say https://t.co/8HqIZsYPli
here is a quick thread for anybody who didn't read the 3 ipcc reports published in the last year (hello, politics, business and news editors) summarizing what hundreds of the world's top scientists concluded in their mega-review of the academic literature on climate change 1/