Introducing: Sawtooth Problems
Why does the Button Dilemma provoke such confusion and animosity between us?
This is the result of a weeks-long descent into mathematical obsession the likes of which made me fear I'd succumbed to AI psychosis.
@mtokson The "little actual evidence" is an extrapolation of the only piece of evidence that seems like it could ever be relevant, though? Like, what other evidence would you expect?
@OlegAlexandrov Yes, and we don't know how to do those things.
Those are unsolved problems which seem to be very difficult. We don't know how to do "goal design." These are hard problems!
It's easy to say "you just have to solve these problems," but it's not easy to actually solve them!
Instrumental convergence isn't an argument about reliability. It's the observation that most goals are served by pursuing certain sub-goals (e.g. "don't get shut down before the goal is fulfilled")
I expect dangerous superintelligence to be extremely reliable before we all die.
@OlegAlexandrov@JeffLadish Given that we don't know what the model weights of an LLM actually encode, I'm confused that you would find instrumental convergence to be a tractable problem.
We quite literally don't know how to predict what an AI will do until we run it, and by then it's too late.
@justMizuyuki I distinctly don't believe that and have a strong aversion to that entire idea.
In the case of Uber surge pricing, it just seems like the situation improves if there are more available Ubers, and offering more money to the drivers seems like a good way to incentivize more Ubers.
@Aella_Girl Having experienced both of these, I think having something stuck in my eye was less painful than a blister from new shoes, but was nevertheless much worse!
As in, it was a less painful experience but the significantly less comfortable experience.
Which would you have me choose?
What he's saying is that "not everyone who needs get it and not everyone who gets it needs it" is the default outcome without price gouging. It's a lottery.
In the Uber scenario, raising the price incentivizes more Uber drivers to go to the airport.
@mmmmccchr3 As someone who probably talks a bit more closely to that roon quote-tweet, I'm not sure what to say except that your alternative example sentence would feel unnatural for me to say.
@OlegAlexandrov@JeffLadish Which of the supporting arguments about the existential threat of AI do you consider weak?
I'm assuming there must be something specific about e.g. instrumental convergence or the orthogonality thesis that you disagree with, right?
@graykevinb I think the $3000 bounty is still up for grabs.
If you believe the rate is 1%, then go gather up a few hundred pre-2020 essays and the prize will be yours.
Very odd that no one's managed to claim it yet, if the false-positive rate is so high.
@MariGO2thepolls@hecubian_devil Fair enough. I appreciate your answer and I think the world would be a better place if more people were this straightforward about their motivations.
@biigrem I don't say it in the sense that "we're supposed to accept it."
I say it in the sense that "if we don't want it to be our future, we need to take political action immediately."
@MariGO2thepolls@hecubian_devil If you believed that dunking on people had a detrimental impact on your policy goals being achieved, even if the impact was small, would you still do it?
@AiKyoiku Get an emergency radio and a lot of drinking water.
But! Those aren't the biggest risks. The risk of human extinction is real, and only international coordination will solve it. Please urge your politicians to call for an international treaty to halt AI capabilities R&D!
@RokoMijic I unplugged my solar electric system from my network. I liked being able to monitor and control it remotely, but this was the only way I could ~guarantee its protection against malicious interference.
The odds seemed pretty small, but the potential consequence was way too high.