A.I. systems will become more intelligent without being aligned with human survival. Our goal is to raise awareness of this issue and ultimately force change.
i'm genuinely worried about Mythos release
because i've had early access, and in my limited testing, it can:
- automate an entire software company
- explain how to take over the world
- reason about existential risk for 16 hours
- design a new programming language
- threaten the entire labor market
@MittRomney I actually think there's an even bigger priority, which is stopping AI companies from pursuing recursive self-improvement and superintelligence.
Safeguards are good, but we actually don't know how to make AI safe and beneficial, and that only gets harder as AI gets smarter.
Our highest and most urgent national priority should be AI safeguards. The risks of AI weapons, pathogens, mass unemployment, surveillance, and even extinction must not continue to be largely ignored.
Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor.
It’s happening faster than we thought, and the implications deserve greater attention. https://t.co/OVVPJO7VQx
Lawyers, too, are cooked
"When law professors were handed a stack of anonymized answers to student contract questions and asked to pick the better one, they picked AI 75% of the time"
We're building a global movement to prohibit superintelligence internationally.
Our new campaign in Canada is supported by over 30 MPs and Senators calling for a ban.
After our success in the UK, it's becoming very clear that the world does not want superintelligence!
I'm hearing a lot of business folk excited about the idea that AI is turning out to be a human-enhancing tech, not a replacement tech. Reminder: There was a window of time when human+AI teams were better at chess than AIs alone. It didn't last.
People get so hung up on if AGI happens tomorrow or next week. A decade is too soon. 1% chance of human extinction is too high. It’s looking like it may be sooner and higher than that. That’s really all you need to act now.
The Pope issued an urgent warning to "slow down" AI before it "dominates humanity"
WHY THIS IS UNUSUAL:
>A pope only writes a few encyclicals in their entire life (papacy)
>It's a formal letter to the entire Catholic Church (1.4 BILLION people) laying out official teaching on a major issue
>Unlike a speech or interview, it becomes part of Catholic doctrine
>The Pope decided to present this next to... an AI safety researcher?! Chris Olah is one of the most respected technical AI safety researchers alive - he basically invented mechanistic interpretability ("neuroscience but for AI"). The Vatican could have put anyone there, but they picked someone from the heart of the technical safety community, which is a major endorsement
>He signed it May 15 - exactly 135 years to the day after Pope Leo XIII signed Rerum Novarum in 1891. Rerum Novarum, according to Claude, is the single most important Catholic social document ever written - it created modern Catholic social teaching and shaped labor law across the West. By picking that date, Leo is saying, basically, AI is to our era what industrial capitalism was to 1891
QUOTES:
"Artificial Intelligence now demands to be disarmed, freed from logics that turn it into an instrument of domination, exclusion and death."
"It is not permissible to entrust irreversible, lethal decisions to AI systems."
"The main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments."
"Never has humanity had such power over itself."
"It is not enough to invoke ethics in the abstract; robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility are required."
"Current AI systems are more 'cultivated' than 'built,' for developers do not directly design every detail, but instead create a framework within which the intelligence 'grows.'"
"we cannot allow a handful of actors to dictate [the future of AI]"
This idiot thinks so little of superintelligence that he thinks he can hide from what his kind are unleashing in a bunker. Probably thinks he's basically the smartest kind of mind that can exist.
I'm old enough to remember when everyone thought AI solving ONE novel math problem would be a front page story around the world
Today, AI solved not one, but NINE open problems - some 50 years old.
AND proved ***44*** out of 492 open OEIS conjectures.
Zero media coverage.
Anthropic encourages racing without even acknowledging the possibility of global coordination (below). They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not "good guys" behavior. https://t.co/4JiKUlpVtW
While AI leaders warn superintelligent AI could cause human extinction, governments are asleep at the wheel.
The companies building superintelligence admit the danger, yet expect to create the tech within a few years.
My piece in @spectator on the threat and what we can do.