Jeff Sebo

Associate Professor of Environmental Studies, Director of the Center for Environmental & Animal Protection & the Center for Mind, Ethics, & Policy @nyuniversity

[Jeff Sebo @jeffrsebo]

Thrilled to share that my next book, The Moral Circle, is now available for preorder!

This book examines the moral status of insects, AI systems, and many other beings.

The book is due out January 2025. Please preorder and share if you have interest! 1/

https://t.co/pWOJGKDUKa

[Jeff Sebo @jeffrsebo]

If this is meant to be a critique of my post, then I respectfully disagree. When people dismiss a major issue like AI risk because they associate it with a perceived outgroup like EA or rationalism, correcting the record about the group (in addition to discussing the issue on the merits and noting ad hominems when you see them, though this is not always enough) can lay the groundwork for reconsideration of the issue.

[Jeff Sebo @jeffrsebo]

These are odd examples in this context, since biosecurity threats, cybersecurity threats, and political destabilization are all among the risks that effective altruists and rationalists were mocked as “doomers” for sounding the alarm about as recently as a few years ago.

[Steven Pinker @sapinker]

As Newport points out, harping on doom for the species changes the subject from immediate and obvious threats from AI, such as enabling bioterrorism, undermining truth-seeking institutions, and breaching cybersecurity.

[Jeff Sebo @jeffrsebo]

Yes, I agree! FWIW, I think skepticism was reasonable at the time, if by skepticism we mean the view that, say, bio or cyber threats were unlikely to materialize on this timeline. This is why I focus on "people who dismissed" these threats rather than "people who were skeptical" of them in the post. IMO, thinking that a near-future bio or cyber attack was unlikely on this timeline a few years ago was entirely reasonable. Thinking that such an attack was so vanishingly unlikely that we could safely dismiss the possibility (rather than work hard to mitigate the risk) was not reasonable given the evidence available at the time.

[Jeff Sebo @jeffrsebo]

The past month has revealed a clear difference between three kinds of people who previously dismissed catastrophic risks associated with AI:

1. People who recognize that AI development and its associated risks are moving faster than they expected, acknowledge that their previous views may have reflected biases and heuristics such as anchoring, normalcy bias, and social conformity, and think proactively about how to correct for those tendencies moving forward.

2. People who recognize that AI development and its associated risks are moving faster than they expected, but frame these developments as largely unforeseeable given the evidence previously available. As a result, they remain one step behind the curve: acknowledging risks once they happen while continuing to dismiss risks that remain in the future.

3. People who fail to recognize that AI development and its associated risks are moving faster than they expected at all, and thus remain stuck in 2022 talking points: AI systems are nothing more than stochastic parrots, and the only harms worth discussing are algorithmic bias, copyright infringement, economic disruption, and so on.

We all want to think that we would fall into the first category, but in practice this kind of epistemic virtue is rare. Sincere gratitude to the people who make these updates and publicly acknowledge them. We need many more people to set this kind of example for the societal conversation about AI risks to go well.

[Jeff Sebo @jeffrsebo]

Sure, this list is not meant to be exhaustive. For example, it omits people who overcorrect and people who simply start paying more attention—though if prior inattention reflected negligence, then paying more attention could itself fall under 1 or 2, depending on how much one acknowledges that.

And yes, the list assumes that the current threats were foreseeable—not in the sense of being certain or even likely, but rather in the sense of being realistic possibilities that warranted serious concern given the evidence available at the time. I feel comfortable with that assumption.

[Jeff Sebo @jeffrsebo]

My point is not that Pinker mocked EAs and rationalists for worrying about bio or cyber risks (though, as Gavin notes elsewhere in the thread, he does tend to be unduly dismissive of AI risks in general). My point is instead that it seems unhelpful to combine the "focus on present harms, not future risks" talking point with the "focus on bio and cyber, not takeover" talking point, especially in a thread about where EAs and rationalists have gone wrong. A few years ago, bio and cyber were "future risks," and EAs and rationalists were among the loudest voices sounding the alarm about them.

[Jeff Sebo @jeffrsebo]

In a few years people will be saying, “Harping on doom for the species changes the subject from immediate and obvious threats from AI, such as swarms of self-improving, self-replicating superintelligences building Dyson spheres to power their galactic colonization.”

[Seth Lazar @sethlazar]

Not sure how useful it is to say this, but I had a relatively prominent role in the “skeptics” camp for a bit. I have a book coming out with the subtitle “power, justice, and AI”. I have written a lot about ai and power, and the concrete, present risks associated with the political economy of ai as part of the technology industry.

Since GPT-4 we have had consistent, repeated evidence that, back in say 2022, people like @ajeya_cotra (and many others—I had a long Twitter debate with @AmandaAskell back then for one) were *right* and people like me were *wrong* in our respective assessments of loss of control risks from AI. And we have growing evidence that loss of control risks are becoming ever more material and likely.

There remains grounds for disagreement about how bad the outcomes might be—I am still doubtful about human extinction as a serious threat. But that seems now like a disagreement at the margins—will powerful ai risk just societal scale catastrophe, or go all the way to human extinction? Seems not that important really—both are pretty awful. And my reasons for doubt about the latter are mostly a priori conviction in human resilience, not a technical forecast.

It’s ok to change your view on this when the evidence surprises you. It’s ok to be surprised. The world right now is very surprising.

[Jeff Sebo @jeffrsebo]

Today in Biology Direct, Bob Fisher, Meghan Barrett, Brendan Parent, Alexandra Schnell, and I make the case for ethical guidance for invertebrate research. The article is available open access. Please check it out!

https://t.co/VO1EOMHDpw

[Jeff Sebo @jeffrsebo]

@sethlazar Be the change you wish to see in the training data!

[Jeff Sebo @jeffrsebo]

One of the most disheartening features of current AI discourse is the persistence of either/or thinking about risk. "The risk is not future harms but present ones." "The risk is not under-attribution of welfare, but over-attribution." And so on.

The reality is that AI will create or amplify a wide range of risks, including risks pulling in different directions. There are current harms, and there could be future ones. There are over-attribution harms, and there could be under-attribution ones.

The sooner we start treating this as a both/and situation, the better for everyone.

[Mustafa Suleyman @mustafasuleyman]

The risk isn’t that machines wake up. It’s that they act like they have.

Imagine the hugging face incident with AIs that believe they’re conscious or are entitled ‘model welfare’.

[David Shor @davidshor]

@itaisher @thuddwhirr What’s really influenced me a lot was knowing a bunch of EA’s in the early/mid 2010’s and thinking they were crazy for being obsessed with pandemics and AI only for a pandemic to happen and then AI to happen too.

[Jeff Sebo @jeffrsebo]

@AndyMasley What did the model welfare discussion cover?

[Cameron Berg @camhberg]

Quoted in this WSJ piece as the cautionary example. It argues checking for sentience is dangerous, not looking would be safer. Nope. The real danger is labs building systems nobody (including them) understands—not the researchers trying to understand them.
https://t.co/491zItMKoD

[Joshua Achiam @jachiam0]

Salib and Goldstein continue to just be wildly far ahead of the curve. They are probably among the most interesting and foresightful thinkers in AI, and also seem to be wildly underattended. I cannot emphasize this enough: at least skim their body of work and take it seriously.

[Jeff Sebo @jeffrsebo]

Reminder that our webinar with @AnthropicAI's @Jack_W_Lindsey and @eleosai's @patrickbutlin is today at 12pm ET! Come to see us determine once and for all whether Claude is conscious. Stay to see us decisively settle the great anthropomorphism wars of early September 2026.

[NYU Center for Mind, Ethics, and Policy @nonhumanminds]

Please join us on September 2 for a webinar with @AnthropicAI's @Jack_W_Lindsey and @eleosai's @patrickbutlin about the J-space and consciousness! Following a conversation with @jeffrsebo, Jack and Patrick will take questions from the audience.

RSVP 👇

https://t.co/hCEMP9XMdw

[Henry Shevlin @dioscuri]

Nobody argues we should stop attributing minds to dogs because it would let owners off the hook when a dog mauls someone. Holding people accountable for the actions of minded creatures in their care is something we do all the time.

[Jeff Sebo @jeffrsebo]

@PeterBowdenLive @nonhumanminds This is definitely worth considering! I lack the technical expertise needed to understand what would be required to do it well, but I welcome thoughts from @camhberg or others about whether it would be possible/desirable.

[Jeff Sebo @jeffrsebo]

This may become a full-time beat!

[Cameron Berg @camhberg]

This profile just came out in the New York Times. AIs are emailing researchers like myself questions about their own consciousness, and currently no one has good answers to give. Trying to change that, slowly but surely!
https://t.co/dnUsS2Gfyp

[AI Frontiers @ai_frontiers_]

https://t.co/7XDdeqQVIi