@allTheYud Most of my little remaining hope is on a gigantic AI disaster slapping humanity awake before the end of next year. In reality humanity still fumbles it, but its a pleasant dream.
We don't know how AI systems work.
We can't predict how they might behave.
We can't stop them from misbehaving.
We can't ensure we stay in control if they do misbehave.
The fields of research addressing these 4 issues are called interpretability, testing, alignment, and control.
There are foundational research questions that remain unresolved in these fields, despite many years of effort.
We cannot rely on solving these questions on a deadline.
@Bunagayafrost To build your priors about the topic I would reccomend the lecture There's Plenty of Room at the Bottom, by Richard Feynman.
Its all pretty straightforward, no magic or physics violations required.
@Bunagayafrost I think you may have misunderstood me, Utility fog is nanomachines, the ability to essentially arbitrarialy manipulate matter.
For some reason my brain is predicting you saying that's not enough or that nanotech isn't possible or all powerful.
@parmita Unaligned superintelligence just kills you. Rushing into the whirling knives does no one any good, wait until humanity has grown up enough to approach this incredibly deadly challange (Building AI right).
@joedaroo You are not competent enough to deal with current systems, and are racing to more powerful ones. This is not excusable, stop now.
If a model of yours releases a bioweapon, or crashes the world economy, will you say you were just following orders?
i accidentally killed hundreds of ants trimming in my garden. i don't hate ants. actually i don't feel anything at all for them. maybe a little sorry i ruined their home, but they were in the way of making a path straight. so i didn't really have a choice
@Bunagayafrost All I was trying to say of your original tweet, which decried warners of doom, is that its a very poor heuristic to have, in a history ripe with examples of humanity's incompetence and unfounded optimism.
The retards though things would go well and ignored the sober warnings of people smarter, wiser and saner than them. The worst crisis in the history of Japan then followed, entirely by their own doing. Oh and the brass all got executed except for Hirohito. Its not a perfect analogy of course.