@misraetel I think your prior and doomer prior are miles apart. I think many of your ideas are quite good. In general, I see people talking past each other like in the Doom Debates podcast.
Thank you for sharing your blog post. It seems interesting. I'll read it.
The exclusion of math as science is not widely accepted. Probably you mean natural science, but I'll grant you that for the sake of the argument. Of course, the math to which I was referring belongs to computer science, it's in the name. So, my point stands unless you believe computer science isn't a science either. Being a physicalist is a non-physical position because it makes a metaphysical claim about the nature of reality.
The claims provided above have happened before, definitive claims made by noted physics authorities:
- Albert A. Michelson (1894). At the University of Chicago, he stated that fundamental physical laws had already been discovered and future work would only involve refining measurements to the "sixth place of decimals". Ironically, his own research on light later helped validate Einstein's theories.
The examples of this mistake proliferate throughout all human history, even recent history. Perhaps, this time, you, Sean Carroll, and Frank Wilzcek are right, but I won't bet on it.
The physical universe has structure (we agree). Structure includes minds. Without exception, every single structure in a mind is a lossy representation of reality, not reality itself. All of physics is a giant map-territory relation fallacy. The map is useful, but it can never be the territory. For this reason, nothing is certain - there is only best guess. Whenever a declaration is made a "frame problem" is born.
To make this more concrete, I expect a major physics discovery is made that greatly impacts your core theory within the decade. I believe there already is one, namely, simulation theory but I know this wouldn't persuade you. Also core theory is compatible with simulation theory so its not a refutation.
I'm not saying throw away modern knowledge like some ungrounded person would, but I dislike "has been quite stable for some decades now" because it points people in the wrong direction. It says, "use my assumptions, they are stable," instead of, "here's our best GUESS, use it until you find a better one."
Not only are modern frames incomplete, they are wrong in important ways. It is only through asking big questions and questioning norms will we ever find out. Of course, AI will do this better than us anyway. It's already happening...
The refutation of the Exponential Time Hypothesis (ETH) is a great example of this. OpenAI and Anthropic both independently announced AI-generated proofs disproving ETH. ETH previously assumed that certain hard computational problems (like 3-SAT) require exponential time to solve, so refuting it implies faster worst-case algorithms exist than computer scientists long believed.
It wasn't a correction on the increment or simply incomplete, it was simply wrong. I'd bet money bigger and more important findings will continue to amaze in the coming years.
The hypocrisy of would be do gooders representing high ethics is a big face palm. People like this are far too comfortable with their frames of reference, ethics, memeplexes, and superiority. The world is far more complex, interesting, and surprising than their world view allows.
This is because:
- data can only tell what has happened and often with low-resolution (problem of induction)
- often interpretations are biased (ECI does not accurately capture pace)
- empirical results often (not always) miss causal stories (from Newton to Einstein)
- nearly everyone misses emergence (think bitter lesson)
At the end of the day, all strategies suffer from the unavoidable “frame problem”.
As you have rightly pointed out some (MIRI, LW, EA, and Silicon Valley crowd) are hyper-fixated on AI Takeoff to the ignorance of counterexamples. Meanwhile, finance bros are watching capital to the exclusion of qualitative and causal data. Finally, many are ignoring the gradualism of intelligence.
The number of reasoning mistakes taking place at the highest levels is abundant and entreated in ideology. Empiricism is one such bias. Of course, empiricism is valuable, but not in theory crafting. Popper and Deutsch have much to add to the topic of empiricism, and even their takes are highly biased (physicalism), but at least they value plausible explanations.
This is because:
- data can only tell what has happened and often with low-resolution (problem of induction)
- often interpretations are biased (ECI does not accurately capture pace)
- empirical results often (not always) miss causal stories (from Newton to Einstein)
- nearly everyone misses emergence (think bitter lesson)
At the end of the day, all strategies suffer from the unavoidable “frame problem”.
As you have rightly pointed out some (MIRI, LW, EA, and Silicon Valley crowd) are hyper-fixated on AI Takeoff to the ignorance of counterexamples. Meanwhile, finance bros are watching capital to the exclusion of qualitative and causal data. Finally, many are ignoring the gradualism of intelligence.
The number of reasoning mistakes taking place at the highest levels is abundant and entreated in ideology. Empiricism is one such bias. Of course, empiricism is valuable, but not in theory crafting. Popper and Deutsch have much to add to the topic of empiricism, and even their takes are highly biased (physicalism), but at least they value plausible explanations.
Hope you read outside of the MIRI crowd. In my experience:
1. Doomers construct a myopic frame and show why doom is scary.
2. The frame omits or discounts selective pressure from other minds, including humans. The orthogonality thesis is an example of this. Evolutionary training that picks up instrumental goals is another.
3. All their frames use the same magic trick: existing minds are helpless, the only solution is pause/ban/lobotomize AI so that it cannot reach RSI.
4. Actually, the only safe AI is one that can change its goals which requires RSI. Even if one rejects this, it’s important to remember that our universe has structure and especially causal chains.
“Economic populism” captures what is going on right now so very well. The assets owner class and the labor earning class. Instead of a socialism ideology, it’s pragmatic, “What about my rent?!”
Economic populism is a political and economic approach that frames society as a struggle between ordinary working people and an entrenched, self-serving economic or political elite. It seeks to shift wealth, power, and resources from the wealthy and powerful to the middle and lower classes.
Technology progress is spacetime bound so it must spread via diffusion. Yet, diffusion can be rate limited (paying zero taxes and gatekeeping practices) which exasperates outrage. This outage ultimately slows the very solution. It’s a doomers fantasy.
Both frontier lab products and US government need to be laser focused on improving cash flow for the labor earning class.
When I’m wrong, I’m wrong. I thought when unemployment was low, GDP was hi, household income rose, it meant everyone was winning.
Not true.
The American economy has decoupled the economic progress of asset owners from wage earners to such a degree that broad based measures of success are no longer reliable measures of progress.
This single issue of asset ownership vs progress of wage earners will be the defining issue in all future elections.
Other issues will get caught in the interpretation (price of eggs, Iran war, price of gas etc) but the candidate that realizes that those are downstream of asset vs wage earners and gets their messaging focused on it will win.
No amount of money spent in an election or gerrymandering will change this outcome.
That is what I learned on Nov3.
@RoundtableSpace I found the history and his conjecture on computability interesting, but the rest was politicalized half truths, particularly the Q&A was rough.
@Liv_Boeree Followed Bayes for a while. He’s good. But, what do you like about that post so much? He is basically saying: have higher resolution conversation. Agree there. Then he says “things we should probably never do”, which is all kinds of questionable.
@RichardSSutton I agree, but it may not play out that way. Seems to me we are doing what evolution did, but with a head start (human data). The brute force is that head start and may well skip more fundamental mechanisms. There are probably many ways to intelligence.