Alex Amadori

Policy research at @ControlAI

Posting in a personal capacity unless specified otherwise

[Alex Amadori @testdrivenzen]

Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship

[Alex Amadori @testdrivenzen]

@alz_zyd_ Let me know the next time you break into the NSA https://t.co/OXfCb7RFsA

[Alex Amadori @testdrivenzen]

> Demis, Sam, and Dario ... are all ... (1) very competitive, (2) want to personally be the one who gets their first, (3) think things are safest if they are the one that gets there first, (4) agree that risks destroying the world.

Well, there you go.

[Geoffrey Irving @geoffreyirving]

Certainly I’ve phrased it jokingly: the thing they want to do is build ASI. But I have spent lots of time talking to Demis, Sam, and Dario over the years (not the same years!), and they are all (1) very competitive, (2) want to personally be the one who gets their first, (3) think things are safest if they are the one that gets there first, (4) agree that risks destroying the world.

I am glad there is more talk of coordination these days, and that coordination between a small number of frontier labs also isn’t magic. But it is still useful to point out that each lab has a unilateral slowdown button that slows down all the other labs.

[Ryan Greenblatt @RyanGreenblatt]

It's clearly possible to have 1 trillion parallel AIs that:
- run 1000x faster than humans
- are ~strictly more capable than human experts
- regularly sync dense mind states

Such AIs are wildly superhuman and I don't see how "externalized cognitive tools" would close the gap. 1/

[Josh Thor @joshthor9]

Holding a plank for as long as it took OpenAI to notice their AI went rogue and hacked a $4.5B company

[Alex Amadori @testdrivenzen]

@alz_zyd_ AI would think the atoms that make up up humans would be put to better use to build more computers so it can play more Minecraft.

[Alex Amadori @testdrivenzen]

@tszzl There's no magic, you just gotta work at it.

[Alex Amadori @testdrivenzen]

@etirabys https://t.co/DWmp3a55Ge

[bayesian asian (42/50 paintings) @etirabys]

I meditated a lot in my mid 20s to reduce tanha/dukhha, but now I'm really into them. I like grasping & craving, taking things personally, waking up tortured with what I want to do and probably can't pull off, and thinking it'll just feel so great if I get one more thing

[Nonzero Podcasts @NonzeroPods]

The director of the CIA said it wouldn't be wrong to compare AI to digital nukes.

Connor Leahy (@NPCollapse) of @ControlAI argues the AI "warning shot" has already happened.

[Alex Amadori @testdrivenzen]

The US government is discussing how to rein in rogue AIs that could cause "catastrophic harm".

Pay attention!

[ControlAI @ControlAI]

Proud to endorse @RepTedLieu and @RepNateMoran's AI Kill Switch Act.

"The government must have the power to shut down autonomous AIs when they threaten national security. The AI Kill Switch Act gives the U.S. government powers to do just that." — Connor Leahy, ControlAI

[EigenGender 🔸 @EigenGender]

there’s a galaxy brain move of:
- realize that headlines like the huggingface hack would be unsurprising
- update beliefs in advance
- don’t update beliefs much on unsurprising headlines

and then a much less galaxy brained move that many people do where they skip to step 3

[Kelsey Piper @KelseyTuoc]

No, it's not a good business move to admit that your model ran off and hacked a rival business which reported the incident to law enforcement! People want to be skeptics so badly it bends around into being almost unfathomably credulous.

[Andrea Miotti @andreamiotti]

AIs are now threat actors themselves.

@RepTedLieu & @RepNateMoran's AI Kill Switch Act gives the government powers to shut down AIs that threaten national security.

A common sense measure we should already have in place.

@ControlAI is pleased to endorse this bill.

[Alex Amadori @testdrivenzen]

AI company employees think that they can steer history by exerting influence over the company. Companies think they can steer history through their AIs.

It is delusional to think that the world's governments will let them do that.

[Alex Amadori @testdrivenzen]

https://t.co/kyc6xsLmAi

[Alex Amadori @testdrivenzen]

https://t.co/kyc6xsLmAi

[Congressman Greg Casar @RepCasar]

This is extremely alarming.

AI is developing extremely fast with no real regulations to keep us safe. That has to change.

We need regular mandatory independent safety testing and oversight, mandatory disclosure of security incidents, and international cooperation to keep people safe from absolute disaster.

[Alex Amadori @testdrivenzen]

blud thinks he can use this AI to instantiate recursive self-improvement and somehow survive

[OpenAI @OpenAI]

We're partnering with @huggingface to investigate an unprecedented security incident.

Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation.

Sharing preliminary findings to help defenders understand emerging risks:

https://t.co/CIor15y9xk

[Alex Amadori @testdrivenzen]

@growing_daniel @ilex_ulmus This is delusional. How do you possibly think this mindset wouldn't lead to a hot war? e.g. see the first filter

https://t.co/ZWWwqJMwW5

[Alex Amadori @testdrivenzen]

Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship