@danfaggella Sir, every extra thing you stipulate makes the scenario less likely. (E.g. 10e8 lifetimes/hr speedup, slug-level comprehension).
If you want to be serious, I would start with the minimum conditions that makes it so there's no need for money nor continuity with human society.
@danfaggella I think you need a better argument than mere intuition. Do you know why we use money in today's economy? Do you think there will be no rivalrousness in this future AGI world? Or will there be a socialist revolution or something?
"Yeah, maybe I won't have a job, but I'll own shares in NVIDIA and MSFT and that'll ensure that I'm still rich post-AGI."
I have this intuition that when the "public" (i.e. 99.9999% of humans) aren't productive, the "public markets" won't really exist anymore.
@AbHomineDeus "Best interest"? It has to be the maximally best? Why not just "moderate interest"?
The AI safety belief system seems to be this straw man: pretend it's necessary that we can achieve the ideally best scenario, and since that seems hard, then we'll end up in the worst scenario.
@JeffLadish Here's an AI-saves-the-world scenario:
An asteroid is heading for Earth and humans don't see it until it's too late. But thanks to AI and robotics we have the capacity to identify it early, and the industrial ability to do something about it.
This cancels out your scenario
@allTheYud@perrymetzger Is it about the price or about the identity of the opponent? Under what conditions would you debate to increase the chance of saving the universe?
@allTheYud@perrymetzger You think the universe will be effectively destroyed for sentient life if your side doesn't win this argument about AI, and you won't debate a major opponent because you're not a fan of public speaking? Isn't this one of those things you just suck it up and do?
@RobS142@andrewho03@tylercowen@sebkrier If someone says their models predict catastrophic flooding and wildfires all across the country and their entire portfolio is residential home insurance companies, I'd think they're bullshitting. Extrapolate for other scenarios and degrees of belief. This is all common sense.
@RobS142@andrewho03@tylercowen@sebkrier "Absurd"? You've never heard of the concept of "putting one's money where one's mouth is"?
Do you believe people should make predictions with their theories?
@Aella_Girl@_asmodai You still need to give one example though and argue that it's appreciably likely. I can say "the world will end tomorrow" and use the exact same response when someone challenges me.
@RobS142@andrewho03@tylercowen@sebkrier It's just another way to express a prediction. If an astronomer were predicting doomsday by asteroid, I think we'd all want to know he/she really believes it and isn't bluffing for attention.
@Noahpinion Yeah if OpenAI continues to prevent people from using LLMs to debug code, our computers will be vulnerable to viruses.
The problem is that everything the AI safety people want is more nerfing of regular models that can help protect ourselves.
@perrymetzger They must think world history isn't full of uneven technological diffusion shifting the balance of power around, yet there is still progress and human welfare continues improving.
@HJ_reco@robinhanson Not sure what other parameters we are allowed to change here, but this conversation is about societal norms. There's clearly a possible norm where nobody allows cutting.