This new UnHerd magazine essay is about the authoritarian politics of AI doomerism.
It explains how it turned philosophical thought experiments into proposals for mass surveillance of researchers.
Now, they're pushing the "AI takeover" talking point.
See, for example, Buck Shlegeris from Redwood Research (the company behind the Hugging Face/OpenAI report):
"If AIs takeover … we might be killed immediately."
They're basically doing the meme.
In a profile article on Berkeley's Redwood Research, METR, and AI Futures Project (AI2027/2040), they called their close-knit group "the Cassandra fringe." They see themselves "like the Trojan princess blessed with powers of prophecy but cursed to watch her warnings go unheeded."
This analogy is revealing, but not in the way intended. Cassandra knew the future; the doomers have no such privilege. They are assigning probabilities to highly speculative futures. The problem is not that the world refuses to heed their prophecy, but the overconfidence with which they treat unverifiable scenarios as grounds for extraordinary authority.
@emollick In the gap there, between 2008 and 2024, is where you'll find the explanation:
The year of the major turning point = 2017.
(The TECHLASH analysis is featured in this book: https://t.co/CrtJvvUA40)
Anthropic complains in its S-1 that its brand "is subject to heightened scrutiny with respect to perceived societal, environmental, labor, health, and safety-related impacts of AI technologies." Those media narratives include that "AI may displace or eliminate jobs, may be misused or otherwise cause harm, and may pose catastrophic or existential risks to humanity."
This "unfavorable media coverage," Anthropic says, "could result in outsized reputational harm" (p. 54), and "could adversely affect our business, financial condition, and results of operations" (p. 83).
Hmmm, hello, not a word about being the main fearmonger of those exact narratives?...
Anthropic put it in its S-1:
"Considerations relating to model safety, interpretability, alignment, and welfare may lead us to limit or delay the deprecation of older models or to continue supporting legacy versions alongside newer releases, creating trade-offs in product development and compute distribution that could adversely affect innovation velocity, infrastructure utilization, and revenue" (p. 29).
Earlier this week, cybersecurity experts discovered something nasty online: two critical flaws in PHP, the code that has been used to build approximately two-thirds of the World Wide Web. On Tuesday, various European governments shut down websites as they scrambled to fix the flaws before malign actors were able to exploit them. Millions more websites will now need to be “patched”. “For the big juicy targets, I hope to goodness they are being patched quickly,” said one cybersecurity expert.
In this case, AI might just have saved the day. The flaws were discovered because cybersecurity experts used AI to search for weaknesses that humans had missed. Had, say, AI-assisted Russian cybercriminals got there first, it might have been far worse. The incident gives an insight into the double-edged nature of AI, which presents clear and present dangers — but also, new tools to combat those dangers.
Note, however, that we are talking about the fairly unsexy business of cybercrime and not, for example, sentient computers spontaneously deciding to eradicate humanity or farming our living souls in some sort of Matrix nightmare. And yet it is these scenarios, dreamed up by a small set of extravagantly funded AI doomers and spread by a credulous media that have occupied the world’s attention; and have gone into overdrive since the Hugging Face hack and Jacob Coxon’s resignation from Anthropic.
Read more from @DrTechlash
https://t.co/oVS2F48hwW
This new UnHerd magazine essay is about the authoritarian politics of AI doomerism.
It explains how it turned philosophical thought experiments into proposals for mass surveillance of researchers.
Journalists have spent years asking readers to imagine the dictatorship of future machines while paying far less attention to the systems of human control being proposed to prevent it.
https://t.co/bNNvXHGNPh
💥 An essay by me for The Telegraph: on the long march of the utilitarians.
People have been focusing on weird and creepy world of the Effective Altruism cult, but the bigger picture is much more disturbing. Does the future belong to Bentham’s Babies?
🎁 https://t.co/EMwP0ksrU6
AI existential-risk ideology did not become a political movement on its own.
Billions in funding helped shape research agendas, career paths, media narratives, and policy proposals.
https://t.co/23veo7eQJA
@Kurz_Gesagt Kurzgesagt got = $7,533,224 from Open Philanthropy
We got = "a virus that only kills people with green eyes," a starving skeleton in front of his screen, and a ridiculous "survival kit" merchandise (to further fund Kurzgesagt).
Cool.
THE ZIZIANS author: "I tracked down sources, interviewed witnesses, uncovered hidden records, and pieced together an impossible story"...
So, yeah, I preordered. Looking forward to reading it.
One of the most shocking and under-reported crime stories of the decade.
@MrAndyNgo exposes the deadly trail of the Zizians in his new book, THE ZIZIANS, available October 13.
Preorder now.
https://t.co/mnt3uoEXDj