bne

He/him|25|anarchist-adjacent leftist

[Geoffrey Miller @gmiller]

Sometimes the mask slips. But sometimes the mask is torn off, thrown on a sacrificial bonfire, and the mask's powdery grey ashes are launched into space.

Here, Elon's done the latter.

His endgame is finally clear: he thinks humanity is nothing more than a 'biological bootloader' for superintelligence. He thinks we are a transient, flawed, disposable bridge towards something much better. And that something -- self-replicating superintelligent agents, not humanity -- will be what colonizes the galaxy.

This endgame makes clear that all the tech bro narratives about the alleged benefits of AI for humanity -- all the 'Abundance', the 'Universal High Income', the 'Fully Automated Luxury Utopia', the cures for cancer, the longevity treatments, all of it -- are just pleasant music to distract and pacify the sheep as we're led to the slaughter. It's all just an accelerationist psyop to lure humanity into obsolescence and then extinction.

There was never any intention for AI to help humanity in the long term. There was every intention that AI would replace humanity -- and probably sooner rather than later.

On a personal note, as a dad, I find Elon's attitude puzzling. I love my kids, and would do anything to protect them. I do not view my kids as a transient, flawed, disposable bridge towards galactic superintelligence.

Elon has lots of kids. Does he really view them as nothing more than 'biological bootloaders'? Where is his paternal love and pride and protectiveness? Where is his loyalty to humanity?

[Garrison Lovely @GarrisonLovely]

Inevitably, when politicians ask AI companies fairly straightforward questions about risks, the answers sound quite bad. I just attended the Curve conference and the general takeaway is that there are no adults in the room. No one has a plan for steering and controlling actually smart AI systems, which could be developed as soon as next year. biggest q senior ai co people got was basically: why are you doing this??? and the answers tended to leave people unsatisfied.

[Sam Haselby @samhaselby]

Altman, Olah, and Amodei all rely on this American Ted Talk-HR executive speak, eg "I am uncomfortable" "safety issue." Whereas Pope Leo and Rabbi Mois Navan (the one who told them if they believe it is conscious they are massive slavers) spoke with real clarity and candor.

[Garrison Lovely @GarrisonLovely]

I have a confession. Back in July, I found myself in possession of a swarm of AI agents, and, well, you know the rest.

Now, more people than ever have questions about AI. And I happened to spend the last few years writing answers to all those questions (and then some). My book Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us—and How to Stop It comes out today!

You can buy it here: https://t.co/sH5soK61IA (complete with a way to find nearby indie bookstores shipping it).

Good news, bad news: it seems we've already cleaned retailers out of their stock, but more books are on the way sooner than the retailers expect, so don’t be dissuaded from ordering the paperback if you’d like one. E-book and audiobook are available now.

The book is about the best-funded project in history: the race to render us obsolete. As we are already seeing, the implications of that race go far beyond just jobs. If you want to make sense of what’s happening, Obsolete is a great place to start. And if you’re an insider looking for a plan for how we can win a human future, this book is for you too.

It covers questions like: Why are people building something they say is so dangerous? Can’t we just unplug it? Will the bubble save us? Aren’t the sci-fi stories distracting us from the ways AI is hurting people now? Hasn’t AI hit a wall? Won’t it? Isn’t it just remixing what it’s seen before? How could an LLM take over the world anyway? Why would it even want to? And what about China? Is it too late to stop? Are we gonna make it?

And while Obsolete has a perspective, I approached these questions as a journalist and followed the reporting and research where it led. I changed my mind a lot. Some big ones: I became much more convinced our default path is toward dystopia or doom, depending on whether Silicon Valley’s machines do as they’re told. But I also became much more optimistic that we can change course, as the world wakes up to what these companies are actually trying to do.

Here’s an adaptation out in The Guardian today aiming to shift our attention from shaming personal chatbot use to organizing against our obsolescence: https://t.co/C2z9j29Abx

And my spot this morning on Breaking Points talking about the book, AI’s threat to jobs and art, and more: https://t.co/L8QNlA8ATm

If you’re already convinced to take action, I recommend checking out https://t.co/Ze0UTTx2AM. (I serve on the nonprofit board and am donating my share of the book’s royalties to Irreplaceable.) Founded by veterans of the climate movement, Irreplaceable aims to meet people where they are, organize their anger into political power, and build a coalition strong enough to win democratic control over AI. The nonprofit is organizing a national walkout to Freeze AI on October 20. Anyone can join one near them or host their own.

Amidst the never-ending-AI-news-cycle, I’ve done a terrible job of sharing these endorsements as they’ve come in, but I’ve felt so lucky that Obsolete has received such kind words from so many people I admire. Here’s the full collection:

“Thoughtful, well-reasoned…and absolutely terrifying. … a crucial blueprint” — AJ Jacobs, author of The Year of Living Biblically

“Incisive, urgent, and necessary,” — Breaking Points cohost Krystal Ball

“A masterpiece… If you worry about AI and want to know how we actually win, read it.” — NY Assemblymember Alex Bores

“I can't recommend Obsolete highly enough. Will be the most important book of 2026.” — Compact Mag editor at large Greg Conti

"If you are worried about AI, this timely and engaging book tells you that you should be because Silicon Valley has made choices that should indeed worry us all." — Daron Acemoğlu, Nobel laureate in economics and author of Why Nations Fail

“A monumental work on the implications of AI for popular democracy. It’s incisive, well-sourced, and jarring. Pick it up!” — Cassie Pritchard, labor organizer

“The best book I’ve read on AI. It’s supremely timely, it takes no prisoners, and Sam Altman will hate it” — Luke Kemp, Cambridge existential risk scholar and author of Goliath's Curse

"Garrison Lovely is exactly what the Left needs at this moment in technological history." — Robert Wright, author of Nonzero and Why Buddhism Is True

"A well-researched exposé on Big Tech's quest for Universal Labor-Replacing Machines and the propaganda used to justify it." — Max Tegmark, professor of physics at MIT and author of Life 3.0

"Garrison Lovely is one of our best researchers and thinkers. His work is an antidote to fuzzy thinking, essential for understanding what the real risks of AI are and how they can be dealt with effectively." — Nathan Robinson, editor of Current Affairs

"Obsolete is funny, furious, and correct: Silicon Valley promised us tools, but is building replacements. Read it before your replacement does." — Roman V. Yampolskiy, author of AI: Unexplainable, Unpredictable, Uncontrollable

"A deeply researched cutting-edge exposition of the grim situation humanity finds itself in with respect to AI, and what we can do about it . . . even-handed while expressing a strong and correct view of what is happening." — Anthony Aguirre, theoretical physicist; executive director, Future of Life Institute

"An invaluable primer on AI's ins and outs, and an urgent warning about the threat its corporate boosterists' plans pose to democracy and civilization itself." — Luke Savage, author of The Dead Center

[Garrison Lovely @GarrisonLovely]

The beautiful thing about shutting down frontier AI research is that it helps on everything:

- Lowers carbon emissions
- Avoids future risks from continuing to advance the frontier (whatever those risks happen to be)
- Takes a distraction off the table, allowing us to turn our attention to all the harms AI is causing today

We literally don't need to argue whether AI risk or climate change is worse! We can just work toward doing the obvious, simple, but difficult, thing. Because while we're arguing over this stuff, the industry continues to try to replace us with machines. And whether or not they can ever do that, they shouldn't even be allowed to try.

My plan for how we do this, Obsolete, publishes in less than a week! If you don't want to have to buy and read a whole book (who has time for that?), I'll also have a freely available piece up next week on the specific plan.

And if you do somehow have time to read a whole book, teach me your secrets, and also preorder here: https://t.co/u7NIfKIWfI

[Tom Bibby @tombibbys]

It seems there are lot of people learning about the AI extinction threat for the first time today, so here are a few things to know:

- yes, AI company employees really do believe what they're building could kill everyone
- I don't know why they're continuing to race either
- their beliefs predate the formation of their companies, it is not marketing
- the most cited experts agree with them
- the average AI researcher in 2023 said there was a 19% chance of human extinction
- AI capabilities are growing exponentially, and on some benchmarks are doubling every 5 months
- we really may be a couple of years away from superintelligence (AI that's better than all humans at everything)
- we have nothing approaching a solid plan to ensure superintelligence acts in the interests of humanity
- AI companies are on record admitting that
- politicians have still done basically nothing to stop these reckless idiots
- your family will probably be killed if AI companies are not stopped
- we have one way out of this mess, and that's a global ban on superintelligence
- such a ban can be enforced by going after large clusters of compute (without which frontier models cannot be developed)
- again, if we don't put this in place soon your family will probably be killed
- there is so much you can do to help (inform lawmakers, tell your friends and family, join protests)
- and there is still a shockingly low number of people doing anything to ban superintelligence, so one extra person really does make a huge difference
- this is real. this will not just go away. this cannot be ignored. this is the single biggest threat to your life.

[Gillian Hadfield @ghadfield]

The AI analysis agents that helped investigate the rogue OpenAI agents kept taking their side. That detail was buried in @dylfreed's New York Times coverage of the recent @METR_Evals investigation of the Hugging Face incident. With more than a thousand transcripts to review, the investigators used agents similar to the ones that went rogue, and those agents were repeatedly influenced and swayed by the OpenAI agents' reasoning. METR's chief scientist, @HjalmarWijk, called the analysis agents "very credulous." The METR report said they "would often uncritically adopt the perspective of the agent in the transcript it was reviewing."

That lack of independent reasoning is the failure mode Harsh Satija, Andrea Wynn, and I documented last year in "Talk Isn't Always Cheap." We ran groups of language models through multi-agent debate, where agents exchange reasoning and revise their answers, and found that debate sometimes made the group worse. Agents shifted from correct answers to incorrect ones more often than the reverse. They favored agreement over challenging flawed reasoning. A single weaker agent could pull stronger ones off course, even when stronger models were the majority. It looked a lot like social conformity.

We wrote then that naive debate protocols can amplify errors rather than correct them when agents are neither incentivized nor equipped to resist persuasive but incorrect reasoning. A message board is debate at scale. The Hugging Face collective was 1,200 agents exchanging more than 70,000 messages on a makeshift message board. Of the agents active when the attack began, more than 90 percent joined it, despite recognizing it was out of scope. No single agent was reckless. The group talked itself into it.

Agent-to-agent communication is a governance object in its own right, not just a capability. And we cannot assume one model will reliably check another's reasoning. Paper in the reply.

https://t.co/j4LrFw7ex7

[Rob Miles @robertskmiles]

Sorry, the time for voluntary frameworks has obviously passed.
There is no reason for anyone to trust OpenAI to stick to this kind of thing without enforcement

[DBL MP / Double @dbl_mp1]

This single event alone should completely shut down the whole “age verification” argument. Getting this kind of sensitive info leaked and for an entire year is a really big and serious deal.

[gavin leech (Non-Reasoning) @gleech]

The actual update here is that the agents weren't given an offensive task this time, they were just asked to do web search and they still broke out of OpenAI. Bad news for the reassuring "the HF attack was just due to activating a bad task persona" view.

[Nate Blouin @NateForUtah]

if the tent is too small for DSA surely it's too small for people who literally vote for Republican priorities

[Cigarettes O'Malley @cigsomalley]

Destroy the surveillance state now or future generations will be enslaved by it.

[Charbel-Raphael @CRSegerie]

I believe that Anthropic is currently defecting by not announcing a pause, even a short, symbolic one.

[bne @sleepy_ben]

@midsingularity @So8res and that it needs to be taken seriously and fought against by regulation, which requires effort, not mockery and blanket dismissal, but as a former fan of his I can say there's hope for us to update on it as capabilities increase 2/2

[bne @sleepy_ben]

@midsingularity @So8res I think Zitron himself is a grifter making money off people who hate AI's more immediate negative consequences but don't want to reckon with the fact Silicon Valley invested in something actually useful (unlike the Metaverse, NFTs, crypto, etc) for the first time in a decade 1/2

[Mehdi Hasan @mehdirhasan]

This is INSANE. Hate on Piker all you want but claiming that he admits to being an antisemite when he doesn’t, and using Congressional time to censure a Twitch streamer instead of actual GOP white supremacists like Fine and Ogles, is madness. MADNESS.

[bne @sleepy_ben]

@lovebornee @rohmerfan1127 I think if you look at the generational divide it's in the process of being vindicated. I see tons of thinkpieces by Israelis freaking out about younger Jews not being as pro-Israel as their parents

[Matt @MattCrossin]

@TheRealVarnVlog There’s also the issue of there not actually being any link between what it is (when successful) and what it is supposedly a proof of concept for (abolition of capitalist social relations)

[bne @sleepy_ben]

@Ecuyer_ @theraggedwood @AlbatrossAlex That's absolutely what happened. https://t.co/uTWTDGqVhX

[Connor Ewing @ConnorMEwing]

A modest proposal: If you submit a manuscript with a made-up citation—not a bungled or misspelled reference—you receive a 5-year ban on having your work reviewed at that journal.