Christoph Bannerman

[Christoph Bannerman @cbannerman]

@__MattPatt__ @BillAckman @POTUS @FLOTUS Angela Merkel did

[Connor Leahy @NPCollapse]

Conrad puts it well. I will elaborate excessively anyways because I am bored:

Normal people have a lot of good intuitions around certain things. Lots of bad intuitions around other things, of course.

You correctly point out that it actually is a strike against people like me and Eliezer that we pattern match to previous (wrong) techno-pessimists. (surface level match of course, if you dig even slightly into our pasts you would find we are/were both avid techno-optimists otherwise, which is not how historical techno-pessimists came about)

But this is a reasonable intuition and signal to have and you should not dismiss it out of hand.

But you also have more intuitions and signal than that. (eventually, if you want to make good decisions, you actually have to reason yourself. alas...)

If your civilization always plays "wait until a disaster happens" with new tech, you are predictably ngmi. Russian Roulette is a great way to make a lot of money, until it very suddenly isn't. You might be right 5/6 of the time, which is more right than the "bullet-pessimist", but so what?

1. The Core Intuition is not stupid

"Creating something smarter (whatever that means) than us, with literally no oversight or even a plan for how to control it, that is supposed to automate/replace all labor - by some of the least responsible people/companies that have hurt me and my family in the past with their products (e.g. facebook), and even despite many experts saying it could be literally the most dangerous thing ever, and even shitting on those experts that are concerned, seems bad" is, in fact, a good intuition.

If you applied this intuition to other situations in the past (and future), you would have been more right than wrong.

2. There is no plan, not even a bad one, for the risks we know will come sooner or later

It's not like there are some expansive, widely discussed plans for how to handle AGI (not just technically, but also how to handle complete collapse of all jobs, how to distribute resources in an automatic economy, how democracy works with digital minds, how to prevent AGI automated giga-war etc etc etc) and we are just quibbling about the technical details.

There is no plan, no one has a plan, not even a bad one.

The intuition that this is concerning is a good intuition.

3. Technology is not a magic paste that makes things better the more of it you apply

Some technologies are actually different from other ones. Nukes are not the same as airplanes, which are not the same as cars, which are not the same as coffee, which is not the same as tissue papers.

You may not be an expert, but I quite trust that your intuitions around the difference between nukes and cars are probably pretty reasonable.

If you treat all of these things exactly the same, your civilization is just ngmi, it's really that simple. It's actually sometimes not that deep.

As we build more and more powerful technology, handling technology carefully and beneficial gets harder, not easier.

Eventually, you have tech so powerful, it can blow up everything. Then what? Continue to refine it until it's mass marketable and accessible? (decentralized maybe???) You may like this aesthetically if you are a libertarian, but ask your intuitions: "Then what happens? What happens if anyone can buy planet busters for 9.99$ on amazon?"

I suspect your intuitions agree that that civilization is ngmi.

And eventually we will have the tech to build 9.99$ planet busters.

Anyone that is trying to sell you that airplanes, coffee and nuclear weapons are in the same reference class is selling you snake oil.

[Christoph Bannerman @cbannerman]

@ScienceGuys_ 24