A Human Future

Stop trying to create a god. Support PauseAI

[A Human Future @A_Human_Future]

Too often missing from the conversation on AI is the fact that Anthropic literally wants to overthrow the government and replace it with AI dictatorship.

[Hayduke ⏹️ (in Bay Area Nov 13-15) @GWHayduke97]

@nihilunbounded Anthropic is particularly unsubtle about this. From a leaked internal slide deck:

[A Human Future @A_Human_Future]

This is all true and important, but I wonder if unionization could help with slowing down RSI. I could see lab employees actually pushing back against RSI because they enjoy their work, and no one likes feeling sidelined and useless. RSI might increase the value of their equity, but also decrease the number of shares they receive.

[A Human Future @A_Human_Future]

@robertwiblin Seems pretty obvious we aren't going to get anything like this complex and nuanced arrangement if we create superintelligence any time soon. We'll either get a durable pause or we'll let Anthropic YOLO us into extinction.

[A Human Future @A_Human_Future]

@RuxandraTeslo I've always found it strange that they are comfortable discussing handing the future over to AI, but none of these people would openly advocate for replacing democracy with a benevolent dictatorship. Apparently the consent of the governed only matters if the government is human?

[Ruxandra Teslo 🧬 @RuxandraTeslo]

I wrote about "handoverism": a vision of permanent disempowerment of humanity, as we shall all be ruled by a (hopefully benevolent) AI God that is surprisingly common.

Even among those who want humanity to survive (which to be clear, is the majority), and among some who devote their working lives to this mission, I have encountered an alarmingly common vision of success that does not sound quite like success.

It is sometimes called “handoverism”: the gist here is that humanity survives, but entrusts the direction of its future entirely to AI. Perhaps its clearest articulation comes from a recent essay by Scott Alexander, the most prominent intellectual in Silicon Valley, who has influenced the thinking of many a tech CEO: “If we build a godlike AI, and determine to our own satisfaction that is fully aligned, I’d rather let it choose its own direction for itself and for humanity [...] So why not just prompt it with ‘please arrange the world as you see fit?’”

The premise is that, once sufficiently reassured of an AI’s benevolence (though it is unclear how we could ever determine that?), we should basically give it control of the workings of society. AI 2040, a proposal written by some of the most prominent members of the AI safety community, offers the most concrete account of handoverism. In this vision, responsibility gradually passes to aligned machines until humans can no longer have influence. As its authors put it: “There is no single moment where humanity relinquishes control. But in theory, there is a point of no return.” Others I have spoken to envisage a much swifter transfer.

This is not an organised movement. I would estimate that something like 40 per cent of the AI safety community outside the labs holds some variation of this view, with a higher proportion of those working inside the labs. Nor is handoverism a coherent doctrine with a clearly articulated political philosophy: it’s more of an aggregation of ideas, a makeshift philosophy so to speak. This is perhaps what one would expect given that humans outside the SF bubble have not really had time to react and come up with good and coherent ideas about what society should look like in the face of the extraordinary pace of AI progress.

The motivations behind handoverism are also extremely varied. Some outright welcome the prospect of a superior intelligence governing our future and consider it an actual positive vision. If I read Alexander correctly, he seems to be in that camp. The enthusiastic handoverist often starts from a very bleak view of what we already have achieved as a civilization. In an earlier, 2014 essay called “Meditations on Moloch,” Alexander argues that competition steadily grinds away what the competitors care about, creating spirals of misaligned incentives. Its ending suggests that the only way to stop this is to instantiate a sort of absolute rule by a benevolent superintelligence.

The second camp are the resigned: they do not necessarily want it, but think it cannot be stopped if we get superintelligence, or prefer it to outright extinction. In my conversations, this resignation has seemed particularly common among people in the AI safety community outside the labs. Inevitability is one very common theme in SF, where many see themselves as mere tools in the unrelenting march of technocapital, a march that can only go in one direction: towards superintelligence. It’s often seen as cool and based to see the starkness of reality for what it is and face it head-on, it is wimpy and loser-coded to try to fight for what many consider the good outcome (which is that humans retain some control and agency.)

Like many loosely defined ideologies, handoverism sits alongside ideas that share its intellectual roots without amounting to quite the same thing. One is the vision of humans as cherished pets, or “labradors,” cared for by intelligences. Yet another related assumption is that human labour will be permanently displaced. This would leave us without an obvious economic role and even perhaps no leverage over our own future or political rights.

To understand how my on-the-ground experience matches with what the lab leaders have said publicly, I asked my “agents” to survey all public statements from the founders of the main labs that are still relevant to the direction of their labs (Mustafa Suleyman is no longer at DeepMind, but is head of Microsoft AI.) You can browse all the findings here.

Some openly support or at least accept outright handoverism. Anthropic’s Tom Brown appears to be most explicitly reconciled to the prospect and he actually uses the term “handoff” in a 2025 interview at Lightcone: “There’s going to be a handoff where humanity will hand off control to transformative AI at some point. And hopefully they’ll be aligned with us and that’ll be a good transition that goes well.”

[A Human Future @A_Human_Future]

@edzitron He's obviously a cokehead.

[A Human Future @A_Human_Future]

@hamandcheese Weird that you think he ever had credibility.

[Alvaro ⏸️ Cuba @GroundhogStrat]

@rutgerbregman Yes! Here is a list of things you can do right now: https://t.co/L7n4WPW7ED

[A Human Future @A_Human_Future]

@littIeramblings I wonder how lab employees feel when they read posts like Buck's. Do they feel remorse? Indifferent? Does it make them feel cool and powerful to be the cause of so much grief and fear?

[A Human Future @A_Human_Future]

@trallala93 At one point or another, all of these guys have said this.

[A Human Future @A_Human_Future]

Too often missing from the conversation on AI is the fact that Anthropic literally wants to overthrow the government and replace it with AI dictatorship.

[Hayduke ⏹️ (in Bay Area Nov 13-15) @GWHayduke97]

@nihilunbounded Anthropic is particularly unsubtle about this. From a leaked internal slide deck:

[A Human Future @A_Human_Future]

@seflapodus I think Musk wants to control it but doesn't really think he can. At least that's what he has been saying on podcasts recently.

[A Human Future @A_Human_Future]

It's interesting and dismaying that the AI lab employees who once were so active in these discussions have seemingly clammed up. Seems like they're resigned to rolling the dice and are unwilling to face the world they've failed.

[Rob Wiblin @robertwiblin]

Paul Christiano (OpenAI board member) yesterday:

"There's material risk of AI takeover in the next 6-12 months.

We have seen roughly the things we might expect to see shortly before something really bad were to happen.

Even with [any] realistic policy responses, the risk remains exceptionally high.

If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].

For most people in this room, the most likely reason they will die is because of an AI accident."

Christiano:

• Is on OpenAI's safety and security committee (which exists to block risky practices)
• Co-invented RLHF at OpenAI back in 2017-2021
• From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI)

(Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)

[A Human Future @A_Human_Future]

@Jason We need to "beat China" by supporting the guy who sold out the USA for a CCP honeypot. Literally the dumbest take imaginable.

[Liv Boeree @Liv_Boeree]

Guys can we not make RSI a partisan issue please? Swarms of hyper autonomous superintelligent agents won’t give a shit about team red *or* team blue.

They’ll be optimising for their goals and their goals alone, which may or may not be compatible with human civilisation.

[Alex Bores @AlexBores]

Every AI company is trying to fully automate AI research; to allow the AIs to build the next AI.

This is called Recursive Self Improvement (RSI).

OpenAI says they are on track to do so by March 2028.

Anthropic employees say they are on track to do so by 2027.

The METR report below suggests they are likely right.

This is a bad idea given current technology. Any mistake or misalignment in current AIs could be multiplied in each generation.

We don't get another election before AI companies reach this threshold.

Congress should ban RSI, this year if possible.

This is a remarkably consensus opinion, supported in various versions by:
-Bernie Sanders
-Steve Bannon
-Ralph Nader
-Glenn Beck
-Republican and Democratic attorneys general
-Whistleblower Daniel Kokotajlo
-Turing Award winner Yoshua Bengio
-Humans First
-Open Markets Institute
-Ezra Klein
-Arvind Narayanan (a Princeton computer scientist known for arguing that “AI is a normal technology”)

The exact mechanism can be debated. The ban doesn't have to be permanent either (see below for both). This is not the only thing we have to do in regards to AI, far from it! But it is urgent.

Congress must ban RSI.

[A Human Future @A_Human_Future]

@ZackKorman They really gulp down any slop he feeds them.

[Zack Korman @ZackKorman]

No matter what Anthropic does, Dario can just write another 10k word essay and at least half the ea / ai safety crowd will be totally on his side again.

Their culture (long form essays) is their kryptonite. They’re powerless against it even when they’re obviously being played.

[A Human Future @A_Human_Future]

@gmiller @TheCoinCollect8 You're replying to a troll account.

[A Human Future @A_Human_Future]

@slatestarcodex @ruben_bloom Is it "illiberal" to try to impose a permanent AI dictatorship on the world, as you've recently advocated?

You really will come up with any excuse for why your friends at Anthropic should be allowed to go full tilt on trying to coup the world with AI.

[A Human Future @A_Human_Future]

@littIeramblings Even Toby Ord appears to be giving up on longer timelines.