I wrote a short book on low-probability, high-stake risks for the Cambridge Elements series. For the next two weeks, it’s free to read and download!
Links + summary below... 🧵
I resigned from Google DeepMind bc it broke its founding promise by selling AI to the military without restrictions against killer robots or mass spying.
For months, I worked to stop this but watched powerful ethicists and institutions choose silence.
Here's what happened. 🧵
New paper: I argue that degrees of value are more fundamental than value comparisons, and that this means we can embrace "individualism" without being committed to principles like Pareto that tie overall comparisons of outcomes/lotteries directly to individual comparisons.
Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor.
It’s happening faster than we thought, and the implications deserve greater attention. https://t.co/OVVPJO7VQx
Our highest and most urgent national priority should be AI safeguards. The risks of AI weapons, pathogens, mass unemployment, surveillance, and even extinction must not continue to be largely ignored.
New Cambridge Element, Small Probabilities and High Stakes, by Christian Tarsney, out now! Read for free for the next 2 weeks at
https://t.co/I3UjxNHq5u
#cambridgeelements#philosophy
The book builds on *lots* of fantastic recent work on this topic, including but not limited to...
https://t.co/J7P1Replni
https://t.co/8izktMHHCn
https://t.co/bAaAu27kWZ
I wrote a short book on low-probability, high-stake risks for the Cambridge Elements series. For the next two weeks, it’s free to read and download!
Links + summary below... 🧵
But (3) we do still have rational permission to deviate from expected value maximization in the *most* extreme low-probability, high-stakes contexts, and so avoid the most implausible implications of pure EV reasoning.
If the world’s leading scientists say there’s even a 10% chance humanity could be destroyed because of uncontrolled AI, shouldn’t we do everything possible to prevent it?
This isn’t about competition with China. It's about coming together to prevent what might be a catastrophe.
I spent the last 2 months trying to prevent this. If OpenAI offered a fig leaf, Google said "imagine we offered a fig leaf."
Google affirms it can't veto usage, commits to modify safety filters at government request, & aspirational language with no legal restrictions.
Shameful.