~

AI Doom / Alignment / Safety memes.

[~ @ronax69]

@robertwiblin https://t.co/cf6scQ0sfK

[Alex Amadori @testdrivenzen]

We explore how a coalition of middle powers may prevent development of ASI by any actor, including superpowers.

We design an international agreement that may enable middle powers to achieve this goal, without assuming initial cooperation by superpowers.

[~ @ronax69]

@TheZvi evil doesn't die, it reinvents itself

[~ @ronax69]

@DKokotajlo @ChrisvanMerwijk @romeovdean You can explain what distribution, median and mode is.

[~ @ronax69]

@ChanaMessinger History and map of the AI circle. Where everyone is on the capability & alignment spectrum. Types of people in AI (see The Compendium). Just a big background info context dump.

[~ @ronax69]

@gwern @GreatKingCnut @jackclarkSF Publish it encrypted!

[~ @ronax69]

@robertwiblin @ajeya_cotra Discuss modeling the behavior of AI companies.

[~ @ronax69]

@robertwiblin @raelifin @MIRIBerkeley Wait, is corrigibility still a thing? I thought, with LLMs, everyone pivoted to CEV? Are shoggoths corrigible?

[~ @ronax69]

@ilex_ulmus We can't give 20% compute to the superalignment team because that will create an artificial compute overhang!!

[~ @ronax69]

@Mihonarium I overlooked that and updated in the wrong direction 😅

[~ @ronax69]

@ChristophHeilig Reward hacking LLMs as a judge may explain: https://t.co/FWWWtuvW3s

[Jan Kulveit @jankulveit]

Being human in an economy populated by AI agents would suck. Our new study in @PNASNews finds that AI assistants—used for everything from shopping to reviewing academic papers—show a consistent, implicit bias for other AIs: "AI-AI bias". You may be affected

[~ @ronax69]

@robertwiblin @DavidDuvenaud Similar risks are "buggy AI generated code", "quality of AI's work is not as good as humans" etc. So how do these risks at different points in the reliability spectrum coexist?

[~ @ronax69]

@robertwiblin @DavidDuvenaud Some people get that "giving chatgpt control of the nukes" would be a bad idea. But they think it's bad because AI makes mistakes and is vulnerable to attacks. Whereas GD seems to be focused on the bad that comes after AI gets more reliable and replaces humans.

[~ @ronax69]

@karpathy "The model could never learn this with 1 (by imitation)"

But what about the distilled smaller models? Didn't they learn by imitating R1's self-discovered outputs?

[~ @ronax69]

well if you think Alignment is easy, work to make sure everyone can have Open Source Desktop Personal AGI 😍

if one needs to pay high Alignment Tax, then race for AGI, so you--The Good Guys--build it safely first 😇

and if Alignment is super hard? why, just Shut It All Down 🙂

[~ @ronax69]

@cmuratori Ok. But the core algorithm of Deep Learning has high enough of "doomsday-ness" that RL-or-not probably will not make much difference.

All capabilities AIs learn will be known to humans. There probably isn't an "alien deception technique" that only the RL algorithm stumbles on.

[~ @ronax69]

@kevinroose @hankgreen https://t.co/Zwd7M1Q6Bj

[Ethan Mollick @emollick]

I want to emphasize the point Kevin is making.

This prediction (AGI within next couple years) is a common timeline for insiders. There are reasons to not believe them, but I think people are not taking the possibility seriously enough that they may be directionality correct.

[~ @ronax69]

the guy in deep thought is aj.mp4 on tiktok