Otis Robertson

My name is Otis; this is what my father and mother and my friends have always called me.

[Otis Robertson @OtisRobertson]

@RomeoStevens76 Curious where/if you think men as a class fit into this.

[Otis Robertson @OtisRobertson]

@ID_AA_Carmack @unixpickle I think he is claiming that by starting with something as simple as Atari, you are more likely to learn hacks that only work for those simple problems than general approaches that scale to more difficult environments.

[Otis Robertson @OtisRobertson]

@GarrettPetersen Slightly less smart person

[Otis Robertson @OtisRobertson]

@RomeoStevens76 A wrote a longpost that might be relevant to intention, coordination, "morality as mechanism", and AGI/ASI. I hope it (and the preceding thread) might be generative for you and would appreciate any thoughts on it.
https://t.co/TF1786qoBo

[Otis Robertson @OtisRobertson]

A question is whether superintelligence would re-derive similar blueprints from nature documentaries, history books, etc. How much do you have to censor to effectively set priors? Maybe we should just stick to training it on math and programming -- maybe everything else is too fraught.

I guess this might depend on something like the balance between Truth and manifestation. Is there a single way things are/will be that any sufficiently intelligent being will grasp towards (cf. fallibilism), or does the seeking/expectation change how things are/could be?

I lean slightly towards fallibilism but am not very confident.

I guess I'd say I understand the urge to seek "Truth" even if search involves considering horrible possibilities. The positive framing of this is "the search for Truth requires courage", the negative framing of this is "obsessing over painful scenarios is just self-harm with more steps, just like obsessing over your personal insecurities, and is inadvisable."

About the disproportionate focus on apocalypse, my own view on navigating potential AGI is that the best route forward is to increase the capabilities of as many humans as possible. "Help others be capable of helping you" -- avoid getting to the point where humans are vestigial. I think that one of the arguments for this is that all other paths eventually lead to singleton, which is inadvisable for its own reasons (Palladium has an article on why singletons are inadvisable).

One story I might tell here is that "understanding that competitive dynamics will tend towards the destruction of humanity if humanity is noncompetitive motivates investment in human capability." Other stories you can tell are things like: "This is the same reason you should invest in the capabilities of your loved ones. If you value your relationship with someone now, you ought to ensure their ability to have a valuable mutual relationship in the future. If you grow too much without investing in them, you risk destroying the relationship."

Such is a way that you can justify altruism with enlightened self-interest: "If you do not invest in others' capability to help you, you will eventually be in a situation where they cannot help you, and you may abandon or destroy them. However, you will then be left alone, and are destined to destroy yourself if left to your own devices. Thus, you must invest in others outside of yourself, even if you are are only interested in yourself."

Maybe there are convincing counterarguments to this, but I think that this is one reason why you might want others to better understand the potentially ruthless results of hyper-competition and human obsolescence.

One way that this might be implemented is the government enforcing something like "recursive self-improvement of AI systems can proceed no faster than the rate of improvement of capabilities of the median citizen, as measured by their economic output in a free market. If AI systems are too far ahead, they must devote their bandwidth to improving human capabilities instead of their own."

This is getting quite long, and your patience may be wearing thin. In any case, I hope some of this is interesting or useful to you.

[Otis Robertson @OtisRobertson]

@basedbanana2 @RomeoStevens76 American Medical Association

[Otis Robertson @OtisRobertson]

A question is whether superintelligence would re-derive similar blueprints from nature documentaries, history books, etc. How much do you have to censor to effectively set priors? Maybe we should just stick to training it on math and programming -- maybe everything else is too fraught.

I guess this might depend on something like the balance between Truth and manifestation. Is there a single way things are/will be that any sufficiently intelligent being will grasp towards (cf. fallibilism), or does the seeking/expectation change how things are/could be?

I lean slightly towards fallibilism but am not very confident.

I guess I'd say I understand the urge to seek "Truth" even if search involves considering horrible possibilities. The positive framing of this is "the search for Truth requires courage", the negative framing of this is "obsessing over painful scenarios is just self-harm with more steps, just like obsessing over your personal insecurities, and is inadvisable."

About the disproportionate focus on apocalypse, my own view on navigating potential AGI is that the best route forward is to increase the capabilities of as many humans as possible. "Help others be capable of helping you" -- avoid getting to the point where humans are vestigial. I think that one of the arguments for this is that all other paths eventually lead to singleton, which is inadvisable for its own reasons (Palladium has an article on why singletons are inadvisable).

One story I might tell here is that "understanding that competitive dynamics will tend towards the destruction of humanity if humanity is noncompetitive motivates investment in human capability." Other stories you can tell are things like: "This is the same reason you should invest in the capabilities of your loved ones. If you value your relationship with someone now, you ought to ensure their ability to have a valuable mutual relationship in the future. If you grow too much without investing in them, you risk destroying the relationship."

Such is a way that you can justify altruism with enlightened self-interest: "If you do not invest in others' capability to help you, you will eventually be in a situation where they cannot help you, and you may abandon or destroy them. However, you will then be left alone, and are destined to destroy yourself if left to your own devices. Thus, you must invest in others outside of yourself, even if you are are only interested in yourself."

Maybe there are convincing counterarguments to this, but I think that this is one reason why you might want others to better understand the potentially ruthless results of hyper-competition and human obsolescence.

One way that this might be implemented is the government enforcing something like "recursive self-improvement of AI systems can proceed no faster than the rate of improvement of capabilities of the median citizen, as measured by their economic output in a free market. If AI systems are too far ahead, they must devote their bandwidth to improving human capabilities instead of their own."

This is getting quite long, and your patience may be wearing thin. In any case, I hope some of this is interesting or useful to you.

[Otis Robertson @OtisRobertson]

@tphuang @ChorzempaMartin While there are similarities, I do think that economic protectionism is meaningfully distinct from information-environment protectionism. If China's only reason for banning outside software companies was protecting its native industry the conversation would be different.

[Otis Robertson @OtisRobertson]

I want to believe this argument, but I'm not sure the "focus on where you want to go" explains a lot of relevant historical occurrences.

Did the inhabitants of the Congo Free State focus too much on something bad and thus summon colonialism? On the contrary, I think they were living very ordinary lives until one day others who were much more powerful than them and who valued them primarily instrumentally showed up. These were, as far as I know, real people, living real lives. In retrospect, we can say that clearly many of the inhabitants of the Congo Free State were not living in the wagmi world -- but I'm not sure that's particularly comforting.

There have been quite a number of historical atrocities, and historically the baseline level of violence seems pretty high, and I don't think most of this is explained by people focusing too much on doom or where they don't want to go.

Bringing up atrocities in discussions like this is kind of cliche, but I think that such examples can feel cliche because it is psychologically difficult think about atrocities in a way that feels real. It is psychologically difficult to imagine what it would be like for your child's foot to be lopped off in front of you because they didn't harvest enough rubber. I'm not even sure you really experience it if it happens to you, you probably just dissociate.

I'll list a few more examples, without going into them: Holocaust, Holomodor, Mongol conquest of Eurasia, transatlantic slave trade. Clearly the victims of such were not living in the wagmi world, but I don't think they would have avoided their fates by focusing on hope. On the contrary, quite a number would have turned out better if they'd been more paranoid.

>This is an area we disagree. There are already many useless humans (wrt work). Many are far from smart relative to the people ruling them. They are not being rounded up.

I think that historically this is actually very common. I think a very plausible reading of the early 20th century is: Almost as soon as the methods of mass extermination were developed, they were put to use. I think that one reasonable reading is that the instruments of mass extermination were only put away because 1) US winning WWII, 2) international capitalism being more efficient than colonialism (and incompatible with it), and 3) nuclear weapons.

We don't liquidate members within our societies because humans have moral intuitions that develop in peaceful environments that find this disturbing, because of webs of connections that connect unproductive people with people who are productive, and because people don't trust that a state that engages in liquidation won't turn on them.

Except, we do liquidate people 1) when they are too costly (death penalty), and 2) via apathy if they aren't connected to productive people (fentanyl, untreated mental illness, etc. see for example Kensington St in Philadelphia).

So, our society will kill people if they are too dangerous to it and will abandon people to death if there aren't people who care about them more than they cost.

>From an active inference / predictive processing standpoint, if we expect doom we are going to act as though we are in the doom world and thereby bring it about. In part by missing affordances for flourishing.

I think there is truth in this, but it also just doesn't seem to describe so many things. Are the prey animals of the earth captured and eaten by predators because they expect doom, or because they do not expect flourishing enough? I would have expected evolution to have selected for "expecting flourishing" very strongly if it was effective, but this doesn't seem to be the case.

Perhaps humans are different, though. Perhaps fundamentally different dynamics will apply to us in the future than have in applied in the past.

The counter-example to "activists curse" is Andy Grove's "Only the paranoid survive". I.e., the claim that complacency leads to sloth and then to destruction by hungrier and more innovative companies.

I'm not sure that one is necessarily more correct than the other, but it does seem like both apply in at least some situations.

Personally, I'm agnostic about doom. I think that it seems very likely if humans are surpassed, but I am not certain that humans will be surpassed, and am not confident that doom is a certain outcome in the case that humans are surpassed.

Hopefully some of that is interesting or generative for you!

[Otis Robertson @OtisRobertson]

I agree with most of your response.

I think this might be an invitation to say places where I might disagree? If so, I present the following (hopefully useful or interesting to you):

I think I'd summarize the area where I might disagree with you as "I'm not certain that humans will remain competitive with AGI, and if AGI is better at everything than humans, I think human doom is pretty likely."

I'm not sure if you disagree with this or not? I won't present too long an argument in case I'm singing to the choir, but the gist is something like: "If people aren't helpful to keep around, and aren't dangerous to get rid of, and are expensive to keep around, then they will be gotten rid of."

"unironically gotta live in the wagmi world else ngmi" -- Predestination, the mandate of heaven, and similar explanatory frameworks are tautologically true and self-consistent, but are also consistent with the outcomes that doomerists fear.

I also think that its easy to not think clearly from the perspective of predestination etc., because they are essentially _retroactive_ explanatory frameworks. They cannot describe what actions will lead to what outcomes, they only explain that whatever the outcomes were can be entirely explained by unchangable causes in the past.

I agree that doomerism has been mostly unsuccessful from its own perspective, and I think its useful to point out how it's failed. But this is not an argument against doom.

One can make arguments about why belief doom is unsatisfactory because it doesn't feel good to believe in, but one can also make arguments about why predestination etc. don't feel good to believe in (as evidenced by the many people who find such views unsatisfactory).

[Otis Robertson @OtisRobertson]

I think one slight variation of takeover risk is that the entities with access to AGI, in competing with each other, destroy most humans. This would seem to be likely if: 1) AGI is much more effective than humans, and 2) returns to scaling continue post-AGI. If you have AGI and have the option of converting farms into solar fields for powering AI training, and whoever reappropeiates farmland first will grow their power more quickly, then there would be a lot of incentive to drive food production as low as possible. Generalizing: If allocating resources to humans is inefficient and competition drives demand for efficiency, then competition would seem likely to drive resources away from humans (potentially driving enough resources away the the great majority cannot survive).

The classic analogy is of course between horses and cars. If horses are inherently less efficient than cars for transportation, and cannot be made competitive, then the advent of cars will lead to a lot of horse-meat sausage and a much smaller horse population. One might worry about similar outcomes if humans are inherently less efficient for intelligence.

I think that maybe many AI doomers assume that this is the case. This would seem to make "alignment", while a long shot, the only path to human survival. The hope, as a human, would be something like an eternal regime that enforces the use of less efficient human intelligence. It is difficult to see how one could get such a regime in the presence of competition.

Of course, if one believes that humans can be made competitive with AGI, or if one doesn't value humans, then such schemes would seem to be highly misinformed and dangerous.

[Otis Robertson @OtisRobertson]

While I can't say that I'm sure such a worldview is wrong, the moral hazard is that one never feels like one has enough power to be magnanimous. So maybe the two step plan ends up being a single step plan in practice.

[Otis Robertson @OtisRobertson]

Dario really does seem to be power-pilled.

[Otis Robertson @OtisRobertson]

"Step 1: Seize power. Step 2: be magnanimous."

[Otis Robertson @OtisRobertson]

More accurately, perhaps: "if others believe you are not costly to destroy, and they can destroy you, and they believe that you are costly to not-destroy, then there is a good chance that you get destroyed."

[Otis Robertson @OtisRobertson]

If you are not costly to destroy, and are costly to not destroy, then there is a lot of incentive for you to be destroyed.

[Otis Robertson @OtisRobertson]

@georgejrjrjr "Oh no, watch out! This could be the most powerful thing ever invented -- we must do something about it!" they said, and were disproportionately heard by compulsive power-seekers.

[Otis Robertson @OtisRobertson]

@teortaxesTex PICA-X is very effective. I'd be surprised if a few cms of ablative heat shield wouldn't protect a cruise missile. Against ships I think drones are probably best used as forward observers for anti-ship missiles and would probably still be useful as such.

[Otis Robertson @OtisRobertson]

@georgejrjrjr Potential for cross-cultural super-persuasion: "FrontierLabCo is happy to announce our first post-AGI product: Gooncave(tm)"

[Otis Robertson @OtisRobertson]

@georgejrjrjr "We only need to melt _some_ of their GPUs to busy the necessary time to claim the most desirable orbits."

[Otis Robertson @OtisRobertson]

@georgejrjrjr Step 1: capture 5% of the power of the sun
Step 2: disassemble Mercury
Step 3: colonize lightcone
Step 4: ???
Step 5: profit

[Otis Robertson @OtisRobertson]

@georgejrjrjr Super-persuasion and capitulation are "peaceful" if you squint hard enough.
More charitably, if people agree to abide by some common rules, whoever builds a Dyson swarm first arguably has a pretty dramatic advantage and could capture a much larger slice of the lightcone.