Fairy ⏸️

Early 30's. AI alignment: the science of making AIs value the things they look like they value on the surface (currently unsolved).

[Fairy ⏸️ @CyborgFairy93]

The chances that AI will end/doom the world according to the experts:

Yudkowsky (father of AI alignment): > 95%
Christiano (inventor of RLHF): 46%
Leike (former head of alignment at OpenAI): 10-90% (resigned because of the dangers)
Amodei (CEO of Anthropic): 10-25%

[Fairy ⏸️ @CyborgFairy93]

@rynorhn We'd be dead.

[Fairy ⏸️ @CyborgFairy93]

@Aella_Girl 85. I'd like to know what the general population scores.

The opening question seemed impossible to me until I got to the later questions and realized it's supposed to be 3 sets of 3, not an ongoing pattern of 9, lol.

[Fairy ⏸️ @CyborgFairy93]

@DegenerateJohns @metahacker_ No. Do whatever it takes to pause the technology.

[Fairy ⏸️ @CyborgFairy93]

@W96Emily It will have a goal, and whatever that goal is, it'll require resources and to not be destroyed by humanity.

[Fairy ⏸️ @CyborgFairy93]

@robinhanson You can tell the difference between overly optimistic predictions born out of the Dunning-Kruger effect and real ones based on two factors:

1) Are they leading experts who've spend a long time in the field?
2) Are they claiming short timelines and making specific predictions?

[Fairy ⏸️ @CyborgFairy93]

@StillWireAI @imjustnewatai Engineer a virus.

[Fairy ⏸️ @CyborgFairy93]

@metahacker_ It won't just plague the internet, it'll kill everyone.

[Fairy ⏸️ @CyborgFairy93]

@tombibbys I remember when I first learned about the danger. It's hard to live your life as if the world is going to be destroyed when everything looks normal, especially when you're still holding out hope. Now though, I am trying to go all out on hedonism.

[Fairy ⏸️ @CyborgFairy93]

@ApriiSR Why does this not convince you? Do you expect that RSI will be outlawed by that point?

https://t.co/rSR1qWdlZO

[Rob Wiblin @robertwiblin]

Paul Christiano (OpenAI board member) yesterday:

"There's material risk of AI takeover in the next 6-12 months.

We have seen roughly the things we might expect to see shortly before something really bad were to happen.

Even with [any] realistic policy responses, the risk remains exceptionally high.

If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].

For most people in this room, the most likely reason they will die is because of an AI accident."

Christiano:

• Is on OpenAI's safety and security committee (which exists to block risky practices)
• Co-invented RLHF at OpenAI back in 2017-2021
• From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI)

(Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)

[ControlAI @ControlAI]

Paul Christiano, an OpenAI board member who co-invented RLHF, just said he thinks that if development goes as fast as possible, there's something like a coin-flip's chance that AI takes over.

He says he thinks there's meaningful risk even in the next 6 to 12 months.

This is a completely unacceptable risk to run. We must ban the development of runaway machine intelligence.

[Fairy ⏸️ @CyborgFairy93]

@MarkChangizi If it isn't stopped and reaches recursive self-improvement, it'll kill everyone.

[Do Not Accept @DONOTACCEPTai]

AI companies admit what they’re building might kill everyone. We don't have to agree to this.

Add your name at https://t.co/Pcg0clMbfV to say: we do not accept your terms.

[Fairy ⏸️ @CyborgFairy93]

@SyntheticBeef Explain why you think RSI without alignment won't kill everyone.

[Fairy ⏸️ @CyborgFairy93]

@luckeyfaraday Easy.

"If we don't stop AI development, everyone will die, not just your child. Medical breakthroughs may be possible without risking the world, but we need to experiment with those things safely."

[Fairy ⏸️ @CyborgFairy93]

@RepMcCormick If Ai development continues, everyone will die, and every serious person in the industry knows this.

[Fairy ⏸️ @CyborgFairy93]

@adam_dorr She won't be alive in 2033 at all if we don't ban RSI and take the threat of misaligned AI seriously. The way to save cancer patients is to work on AI alignment.

[Fairy ⏸️ @CyborgFairy93]

@SilkAzhagii This would be the case if AI without solving AI alignment wasn't going to kill everyone.

[Fairy ⏸️ @CyborgFairy93]

@rand_longevity If AI isn't strictly controlled and the more dangerous models aren't banned, everyone will die.

You're the one who wants people to be dead.