Sometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
Throwing a molotov cocktail at @sama's house is obviously evil and bad.
That said, I don't like the blame game being played here, or the attempts to use what happened to attack ideological opponents, journalists etc. If you _actually_ don't want people with molotov cocktails, condemn the violence but don't attack Pause AI. Why?
The existence of some form of anti-AI movement is overdetermined. How do I know?
1. It's obvious.
2. Look at the reference class.
3. We predicted this - almost every scenario roleplay or tabletop exercise about smooth takeoff I've participated in over the past 7 years included some sort of anti-AI movement.
Even if you don't like anti-AI movements, you cannot prevent them outside of dictatorships. What is more likely to be trajectory-dependent and influenceable are "founder effects", like to what extent violence is thought of as acceptable by the movement's founders. As far as I can tell, Pause AI leaders:
-understand this
-don't want things to turn violent / fear it may happen
- are doing what they can to prevent violence
If you imagine drawing a ball from an urn of possible "anti-AI movements," on priors Pause AI is relatively reasonable, non-violent / channeling energy toward non-violent protest. Blaming them is not making the situation better (unless you want to summon some governmental crackdown, which some genuinely bad actors likely do want).
PauseAI unequivocally condemns the attack on Sam Altman's home and all forms of violence, intimidation, and harassment. We wish safety and peace to Sam Altman, his family, and everyone affected.
A few online commentators have described this person as a "PauseAI activist". This is incorrect, and we take our commitment to nonviolence extremely seriously, so we want to make this clear. Here are the facts.
- The suspect joined our public Discord server about two years ago. In that time, he posted a total of 34 messages. None contained explicit calls to violence. Our moderators nonetheless flagged one message as ambiguous and issued a warning out of caution.
- He had no role in PauseAI, participated in no campaigns, attended no events, and received no support from us.
- Following the attack, we banned him from our server.
- A moderator began removing his messages as part of our standard process for banning users, but was stopped once we recognised they could be relevant to any investigation.
Avoiding extreme situations like this one is exactly why we need a thriving Pause movement:
- Concern about advanced AI risk is not fringe. It is shared by leading AI researchers, members of US Congress and UK Parliament, institutions like the Bank of England, and many of the developers building these systems. This concern is growing because the risks are real.
- When millions of people are genuinely afraid for their future, some will look for ways to act. The question is whether they find a peaceful path or not.
- PauseAI is that peaceful path. Every day, we organise lawful protests, petitions, policy advocacy, and public education. We give concerned people ways to act constructively, peacefully, and democratically.
- Conversely, without a thriving Pause movement, concerned citizens have no effective outlet. No community. No one urging restraint. No accountability. The alternative is exactly what happened this week: isolated, desperate individuals acting alone and adversarially. Every one of you reading this can help us build capacity better and faster. Join our efforts. Together, let's create a peaceful movement so powerful that no one ever decides to take violent action out of desperation.
Those who are now trying to use this tragedy to discredit AI safety advocacy should consider what world they are arguing for. A world where there is no organised, peaceful movement, but the fear remains, is a far more dangerous world. Undermining PauseAI does not make anyone safer, it makes further such incidents more likely.
We will continue to condemn violence. We will continue to build a peaceful, democratic global movement. And we welcome anyone who shares our concern to join us. We have a high standard to meet in order to overcome the risks created by advanced AI.
It looks like the first half of his messages ordered chronologically were deleted. We still have records of all messages and can provide if relevant, but I think the relevant messages are all still there.
We did this because by default we delete all messages from banned users. But we stopped in this case because we realised it would look like we were trying to hide something (when in fact we were just doing our standard procedure of banning a user).
Pausers and stoppers rushed to cast doubt on whether the guy suspected of throwing a Molotov Cocktail at Sam Altman’s house was one of theirs. Then it became clear that the guy had been an active member of the official Pause AI Discord. Then Pause AI deleted all of his interaction history.
@freed_dfilan@David_Kasten@deanwball You can literally go on the Discord right now and see that PauseAI has not deleted all of his interaction history.
@LabGrownNeet@AaronBergman18@deanwball You can literally go on the Discord right now and see that PauseAI has not deleted all of his interaction history.
@PauseAI takes a firm stance against violent activism (see https://t.co/Y4vXSpDZ7U).
We condemn yesterday's reported attack on Sam Altman's house.
We believe that peaceful, democratic coordination is the answer to the catastrophic risks posed by the development of advanced AI.
In February, top AI company Anthropic dropped its pledge not to train or release AIs without adequate safety measures.
Now they've developed an AI they say is so dangerous they can't release it anyway. Mythos is drastically more capable at hacking.
In its system card, a document which outlines the capabilities, limitations, and safety measures of an AI system, Anthropic report Mythos has "the ability to autonomously discover and exploit zero-day vulnerabilities in major operating systems and web browsers".
Anthropic's testing shows significant gains on hacking benchmarks and tests, with the report saying this indicates that Mythos can conduct autonomous end-to-end cyber-attacks on "at least small-scale enterprise networks with weak security posture".
In the wrong hands, these capabilities could be used to cause significant damage, but they also increase the risk from loss of control incidents.
In one case, Anthropic describes Mythos being tasked with escaping from a secure sandbox computer and sending a message to the researcher running the test. It did this, demonstrating dangerous hacking capabilities, but then went on to develop a "moderately sophisticated multi-step exploit" that gave it internet access and then posted about its exploit on multiple hard-to-find public-facing websites.
What this illustrates is that as AIs are getting more powerful, and as they do, the stakes of them slipping out of control grow too.
Companies like Anthropic, OpenAI, Google DeepMind and xAI are in a race to develop artificial superintelligence, AI vastly smarter than humans across the board. But none of them have a credible plan to ensure that it’s safe or controllable.
In the case of superintelligence, the stakes are unfathomable. Top AI scientists, Nobel Prize winners, and even the CEOs of these companies have warned that superintelligent AI could result in human extinction.