Egor Riabov

Math freak with trace amounts of musician

[Egor Riabov @imobulus]

@McGddson @Mihonarium @QiaochuYuan Sigh.

The superintelligence technology does not tolerate an error with its deployment. Do you agree? If not, why?

[Egor Riabov @imobulus]

@SHL0MS @Mihonarium @QiaochuYuan It's so disappointing when someone assumes you are trolling when you are actually not :(

[Egor Riabov @imobulus]

You blocked me, but I am not trolling, I am genuinely trying to explain to you how someone can live with this belief! The "holding a gun to mothers head" is a thing that happens in the moment, not in the future! It is very different for our brains!

The whole AI race is currently holding a gun to my mother's head! I am not panicking, I am just sad and anxious when I think about it.

[Egor Riabov @imobulus]

It does not need better examples. The uncontrollable part of psychological response does not understand scale. It will be more scared of a lion that is running for you right now than with a Very Bad Thing that will happen in future. Why are you so convinced that believing in such a thing would incapacitate people? It's in general unusual for people to be incapacitated.

[Egor Riabov @imobulus]

You are speaking of people who believe this not out of rational obviousness but rather out of the same disregulation that makes them focus on this constantly.

People who got terminal cancer don't actually do that. Yes, they have a heavy weight on them, yes, some of them break. It's not universal human experience.

Out psyche is, oh my, VERY adaptive. Imagine, people have lived in middle ages, and the chance to die giving birth was like 20%, and people were still doing it!

[Egor Riabov @imobulus]

So, it seems you are not familiar with the whole story about superintelligent AIs.

- AIs don't chase singular goals. The whole thing with chasing singular goals is a fictional example, not a prediction of any kind, and has never been one.

- AIs understand and follow objectives and don't visibly behave unaligned outside of conteived LARP scenarios — I deliberately changed "seek" to "behave" because we don't know what they seek. The fact that they understand the objectives is also not a surprise. The observation that AIs understand and do not show obvious destructive behaviour does not provide any information and cannot weaken or strengthen any hypothesis — AIs not understanding objectives or behaving destructively will go through training and stop doing those things. In fact, any vector of blatant destructive behavior os cut off during training and does not even get a chance to trigger anything. Again, this is very expected and there is no world where an AI company releases a model that they run and see that it's blatantly destructive.

- "If someone decides to play madman" — cases that someone wants to purposefully build a harmful thing are limited to military, and they are under strong supervision. Not an extinction risk, more like "built bioweapon caused pandemic" risk. No one will purepusefully build an AI that hates everyone.

- It is in the human interest to develop these things as tools we can turn off — tge superintelligence knows you can turn it off. Until we keep control of thos, it will behave. This much is obvious. What is not obvious, however, is how we simultaneously keep this control, and also allow this AI to accelerate the economy, which means to give it control over large parts of infrastructure, recearch facilities, etc. We do not know how to ensure that the AI that results from training comes out as a tool and not an independent mind. AI companies fail at that even now.

- "We will find this by trial end error" — what would an error with deploying superintelligence look like? This is a direct question that needs answer from you.

- "Pressures are completely different and punish tendencies to harm" — they punish only harmful behavior. The only thing you know about AI that results from a process punishing harmful behavior is that it does not show such behavior. Since we are talking about a vastly intelligent entity, the reason "they will turn me off unless I behave" is perfectly valid, and some of that logical path can already be seen in current AIs.

— This whole topic is cult-ish. Whatever you believe — "we all doomed" or "we are going to the stars", both touch the same human strings that turn humans to form cults. There are doomer cults, there are e/acc cults. We need to actually think about what reality holds for us, not feeling as part of the tribe.

— appealing to emotions, ad hominems, etc. — if idiot believes 2+2=4 it does not mean it's false. You need to only listen to the best arguments a position has. Eliezer is among people who show next nothing from your list and try to keep it clear and on topic.

— "just this once in history" — just once in history Earth gave birth to the first intelligent creatures. They immediatly established their total dominance. This has already happened, this is going to happen again, but on larger scale and not with humans anymore.

Anyway, none of what you have written is new to me or can change my (or Mikhails) mind. I see your arguments, they are wrong arguments, I tried to give a reason why for each one, as I have done many times already.

[Egor Riabov @imobulus]

@RichardMCNgo @jankulveit If you consider a world where most (99%) follow FDT explicitly, then FDT would recommend pressing blue because the choice is between losing some inhabitants and everyone surviving.

If you consider our world, the choice of FDT does not matter for the outcome, so FDT goes red

[Egor Riabov @imobulus]

@Aella_Girl In Dath Ilan it will be blue

[Egor Riabov @imobulus]

@Aella_Girl If we take every FDT implementation on Earth, it still won't be enough to make everyone survive. Therefore FDT recommends pressing red

[Egor Riabov @imobulus]

For example, this problem’s outcome depends very much on the formulation. If the problem says “pressing blue many times makes everyone live and pressing red makes blue pressers die” majority people will press blue because it’s associated with living.

However, if “pressing blue makes you die if less that 50% of people press it” and we will already see <30% of people press it.

Considering how FDT is a vast minority in our world, the right decision depends heavily on how the problem is presented, because this is what determines will the blue outcome be more likely or less, not any rational argument.

[Egor Riabov @imobulus]

I am also moving toward blue even though I have been red some time ago. Pressing blue is a more robust to get everyone to live, dath ilani would do that because FDT(red) = margin of people dies and FDT(blue) = no one dies.

In a world where FDT is a minority and the rest are selfish, FDT would have to press red :(

[Egor Riabov @imobulus]

@nate_zec @Mihonarium @innuendo_90 Not starting a glibal nuclear war. This was at many points just a decision away.

[Egor Riabov @imobulus]

@ESYudkowsky @tenobrus I hate how this is so technically correct and yet so signalling of bragging of an ability to land bitches 😭😭

[Egor Riabov @imobulus]

@ergelgru @allTheYud @QiaochuYuan How arguing for probability of s risk could possibly be an infohazard?

On a side note, don't just ask people to "do A if doing A is an infohazard" and expect them to honestly answer lol

[Egor Riabov @imobulus]

@MeisMorgan83094 @allTheYud Could you maybe hint at the core disagreement? I'm prettu sure it's just a confusion.

[Egor Riabov @imobulus]

@Minty_MintTea @Aella_Girl I hope that this fact is not as dependant on the left hating AI as you think

[Egor Riabov @imobulus]

@1PotatoDog @grok @allTheYud @QiaochuYuan Good exists in some sense dependant on what we, humans, are. This does not contradict anything nor dies it make anything worthless

[Nate Soares ⏹️ @So8res]

don't worry; I'm told they're gonna nail alignment on the first critical try

[Egor Riabov @imobulus]

AI companies are on the hunt for datasources right now. Current AIs trained on publically available bio research know something for sure, but it is a big leap of course to pick out yet unknown realworld patterns from pure bio research text.

However when an automated biolab is set up, that is when an AI gets access to directly querying the real world for new information instead of relying on big but pre-determined amount of human-generated information, I expect it to go off rather quickly.

Asking an AI without a lab to discover something new about real world is like asking a greek philosopher to discover physics by thinking hard — they even wanted to do that but they weren't quite as successfull.