Bernie Sanders calls for a Pause in AI development. Let's make it a bipartisan ask quickly.
Please repost. It's time to start hammering the reposts, people.
@PauseAI
"This is not an economic goal, it is a political goal."
Our US Executive Director Connor Leahy (@NPCollapse) on AI companies' real pitch.
Superintelligence isn't about 20% returns. It's the power to replace the government, replace all workers, and ultimately, replace humans.
@jmrphy Unrestrained capitalism could kill everyone. We can just stop AI development and let capitalism continue elsewhere. Imo socialism would be throwing out the baby with the bathwater.
The main case for building datacenters to the extent we are seeing now is to train and run inference for extremely powerful AI models, namely AGIs and ASIs (both of which are the explicit end goal of the two biggest AI labs who are driving most of the datacenter build out, Anthropic and OpenAI). Such systems, if they exist and are practical to deploy at scale, would outcompete humans at much of the work relevant to the economy. If allowed to grow to sufficient number with direct control of the right machinery, they would also be fully capable of killing us if they desired it. When polled AI researchers think this specific outcome is like 5-15% likely. This is not to mention the AIs killing or disenfranchising us not of their own volition but on the orders of a small number of humans who own them. But of course it is not certain we will achieve AGI or ASI, in which case a buildout of this scale is much less justified.
All in all I think there is extreme uncertainty as to whether or not AI is net positive for the everyday person. This makes it weird to think that building datacenters is unambiguously good. Most arguments against datacenters are deeply stupid but it doesn’t follow that just because those are fallacious it means datacenters have great expected value for us.
BERNIE SANDERS DEMANDS AI COMPANIES PAUSE - OR ELSE
"Mr. Altman, Mr. Amodei and Mr. Zuckerberg:
Stop building machines that humans cannot control.
Let me be very clear: If you do not take action now, my colleagues and I in the U.S. Senate will.
It is not too late to avoid disaster.
In the interest of humanity, stand by your words. Pause AI development.
Almost every day, there is a new story about how your companies are losing control of the AI technology you are developing, with potentially cataclysmic results.
This week we learned, frighteningly, that AI has been used for the first time ever to create new viruses. As you know this type of development, in the wrong hands, could lead to new bioweapons that result in the deaths of tens of millions of people.
Last month, the world found out OpenAI lost control of an AI model. The result? The model hacked into another company’s computers—a clear violation of federal law. After conducting internal reviews, Anthropic and Meta reported their models similarly escaped their control.
One of the targeted companies called the AI hack “an unprecedented event” that deserves an “unprecedented response.” Yoshua Bengio, the most cited living scientist in the world, said these incidents “should serve as a wake-up call.”
I agree.
So do the top scientists at the companies you lead, the very people building this technology. As you know, these technology leaders recently called for the international community to create a safety mechanism—a pause button—to avoid catastrophe. They warned there is a “real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.”
And yet, at a moment when we have seen human loss of control and the creation of potentially dangerous viruses, your companies are still racing ahead— investing tens of billions of dollars into a technology that nobody can fully understand, predict or control.
That is absurd, irresponsible and extremely dangerous. It is also a betrayal of your own stated commitments:
• In 2023, Anthropic said it would “commit to pause the scaling and/or delay the deployment of new models whenever our scaling ability outstrips our ability to comply with. . . safety procedures.”
• In 2025, Meta said “if a frontier AI is assessed to have reached the critical risk threshold and cannot be mitigated, we will stop development.”
• That same year, OpenAI said it would “halt further development” until strong safeguards were in place if AI capabilities ever reached a “critical” threshold.
That moment is here. AI capabilities HAVE reached a critical threshold. There is a reason why the head of the CIA says that AI models are “akin to digital nuclear weapons” and “almost like a doomsday device.”
Very much worth your time.
TL/DR: An actual post-ASI, robot economy future is:
—Zero-growth (there’s no demand or dynamism to drive further growth, so it likely caps out)
—Neofeudal (you can’t better your station by work, so the social hierarchy freezes in place)
—Patrimonial (resembling the Gulf states, where citizens are permanently on the dole from a centralized state economy - an analogy I’ve made in the past)
—Totalitarian in nature (the tech offers total legibility and surveillance)
—Totalitarian in scope (you can be rendered dangerously poor just by being cut off from state or corporate charity)
I differ from the author in his assessment that this is likely, given a number of political and technological realities. Of note:
The ability of the PMC (“professional/managerial class” - what I would call the petty elite) to entrench themselves in make-work jobs is demonstrated (in their feud with Elon over DOGE, Elon was forced to back off). They will fight back. And without buy-in from some part of them, it’s politically dicey to try to implement this program.
Moreover, there are so many jobs that very obviously cannot be done efficiently by robots, and many more that cannot be done -reliably- by them, that even the advent of some type of ASI probably would not make for a robot future. Yes, an AI could theoretically fly a plane, but it would require a completely different regulatory environment to make that even theoretically legally acceptable, and it’s still not a good idea. And while it’s theoretically possible to imagine a robot plumber or electrician, it’s not obvious why this would be a cost-effective improvement. At the end of the day, even granting unprecedented monopolistic control over the marketplace, somebody has to want to buy this stuff.
And, while the article necessarily handwaves the AI safety issue, in the real world, it cannot be ignored for much longer. The inability of AI companies to control or “align” frontier models, and chains of agents based on them, is now an open secret - the likelihood of a mishap multiplies exponentially the more robots and agents are out there. This is likely to go critical: Even if the companies are not held to account legally, and even if no AI safety regime emerges, they themselves will eventually be hoisted by their own software petards if they do not learn to tame the monsters they have created.
But the article is an excellent summation of what the tech bros -want to happen-, and it goes to illustrate a point I make frequently: If this stuff works as intended, it is an assault on civilization and humanity. It’s only if it doesn’t that we merely have to deal with educational and civil decay resulting from the less carnivorous version.
But it’s difficult to see anything good coming of this.
And whenever anybody says we need to build this to “beat China,” -this- 👆 is what they mean. It’s an open question what would survive of American culture, law, ideals, or civil society if this program were implemented, and the idea that such a society would command any willing loyalty should be seen as farcical.
“It became necessary to destroy America to beat China” should not be the final word in a history of this era (that no human American ever writes).
As for my own thoughts: I’ve said it often - if you need this stuff to drive (to date largely nonexistent) drones or missiles, it should be sequestered in a government installation, attended and maintained by dispassionate, loyal and hopefully politically ~neutral personnel, and carefully monitored.
And if in the process we discover it still has a mind of its own, we’ll at least be in a position to shut it down.
Pretty much every single new paper on AI and learning in the last year says the same thing: "AI gives the illusion of learning in the short term but leads to less learning in the long-term." If you are giving students access to raw chatbots then what do you expect?
https://t.co/X3UTbnygs3
The people most closely involved in building AI are living on another planet.
The vast vast majority of people in the world do not want any of this. Even the things that the AI avant-garde think are positives are, for most people, actually negatives.
“[AI] risks becoming an iron curtain that comes down between a young person and their potential for thinking… it risks halting learning before it can take flight.”
Moving and important article by the great Katherine Rundell. Please read!
https://t.co/jECfE2cZOi
Seems like a fucking good time to read IF ANYONE BUILDS IT, EVERYONE DIES. Or the free three-volume sequel series we shadow-released at the same time as the book (branded as the book's "online resources").