A big issue w the AI doomer crowd is that many of them are experiencing deep psychological issues similar to others in doomsday cults. They live in abject panic & fear, convinced AI will murder their loved ones. I worry that as their fears intensify things will turn violent
Even if it's not computable, it seems that in principle, it could still provide a useful yardstick in some discussions. For instance, if you can show that even in that ideal system, some of the things that people expect ASI to do are impossible, the fact that no actual AI system will ever achieve the capabilities of that ideal system doesn't change the fact that it's a useful result. I don't know to what extent this theoretical possibility arises in practice though.
By the way, I was surprised to read in the latest DeepMind essay on the road from AGI to ASI this passage, which directly contradicts what I said in my tweet.
The book they cite in support of their claim here is An Introduction to Universal Artificial Intelligence by Marcus Hutter.
The central model in the book is apparently AIXI, which I remember gwern mentioning in one of his posts, but I'm wondering if anyone has read it and whether they think it's worth going through it.
Critics of IQ like to point out that we don't have a well-developed theory of intelligence. I don't think that's a good reason to reject IQ, but it's still true in a very real sense and I think there is a good case to be made that it's partly why we don't have a good sense of what to expect with AI.
For instance, a lot of the debates about ASI seems to hinge on whether there is a "natural" bound to how smart something can be and, if so, how far humans are from that limit, but as far as I can tell we don't really know and it looks as though we're going to have to find out in the dumb way.
Maps of the modern greening of the Sahel. Probably the biggest overlooked topic in Anglosphere geography departments. Most of the relevant fieldwork, on both ancient Green Sahara and emerging new greenbelts, is from the French.
I haven’t read the book yet, but just briefly asking ChatGPT about it, it seems that it could be a useful model to derive results about the theoretical limits of intelligence, but 1) I don’t know to what extent the model is a good model for our pre-theoretical concept of intelligence and 2) it’s not clear that it has much bearing on any actual AI system.
Yes, I'm pretty sure gwern said that in the post I have in mind, but I still think it could be useful to have a theoretical optimum against which to measure expectations. I think the book also talks about computable approximations to AIXI, but I don't know how relevant the results derived from the ideal model are to them.
(Which to be clear wasn't a crazy assumption, I think it's basically impossible to predict that kind of morale-related thing and I still wonder whether, in a not very distant possible world, Russia's initial assault wouldn't have worked because just a few people in Ukraine got discouraged instead of what happened in the actual world and now everybody would be jerking off about how all-powerful the Russian military was, even though success would have been down to a few chance events.)
I don't know, maybe they thought Russia could assassinate Zelensky and the rest of the leadership and had good reasons to think it could pull it off, but they seem to have believed its attack on Kiev had a high chance of succeeding and that's what I don't understand in retrospect.
I remember thinking that it would fall quickly at the very beginning purely based on press reports that Milley had told Congress in a private hearing it was likely, but now that I have read about modern warfare, I don't really get it.
The thing that impressed me the most is tables in books I read that show the logistical requirements of keeping large units operational. They are really enormous and other tables show you how much stuff you can carry by truck and it's just minuscule. So I don't really get how it was supposed to work unless the assumption was that the Ukrainians would not fight.
If you look at contemporary press reports, well into March, the US government was still planning to support a Ukrainian government in exile. Most governments seem to have assumed that Ukraine would collapse relatively quickly, which is also what I assumed at the time, but unlike the people advising Western governments (presumably), I didn't know anything about modern warfare at the time.
I still don't know all that much about it, but I have read a decent amount about it to make up for this deficiency since then, and it has actually made me puzzled that Western government experts were seemingly confident Ukraine would collapse militarily, because learning more about modern warfare has actually convinced me that it was really difficult to pull this off with the resources Russia initially committed. I guess maybe they thought the Ukrainians would be discouraged and not fight.
I think people greatly overestimate how much the success of Ukraine's initial resistance owed to Western aid, which is very little in my opinion, but greatly underestimate the extent to which the success of its continued resistance after that owed to Western assistance.
I also think they don't appreciate the extent to which the West developed a commitment to Ukraine's defense, not so much out of enthusiasm for making such a commitment, but because once Ukraine was able to resist the initial assault it became politically, I'm using the word in a broad sense to include internal bureaucratic pressure, very difficult not to make such a commitment.
It’s much more reliant on Western military aid now than it was at the beginning and, despite the initial failure of the Russian invasion (which I agree didn’t have much to do with Western support), would have militarily collapsed within a few months for lack of ammunition without it.
Even putting aside Western military aid today, were it not for Western economic assistance, it would be incapable of sustaining its war economy because its budget would collapse almost immediately.
Obviously, it’s still evidence of Russia’s relative weakness that it couldn’t prevent things from getting to this point, but let’s not exaggerate. I think it’s just as much evidence of the fact that defeating a country like Ukraine with a large and relatively competent military is actually very hard.
This note from Bao Chau Ngo (Fields Medal 2010, Clay Research Award) on Alpoge's work for Riemann is incredible. It's rare to see a mathematician of his caliber with such an open mind about AI.
This and the fact that I expect capex plans to be cut when people realize that revenue will not be growing fast enough for the investment to earn a reasonable return is specifically why I talked about a scenario where spending on hardware doesn’t grow, although to be clear I think it will grow even if not as much as anticipated right now and physical bottlenecks in the industry will not prevent that because there are already significant investments in increasing production capacities under way right now.
This chart is kind of mind-blowing when you think about the likely implications of what it shows by the way.
Even if you assume that hardware spend will stop growing, if everything else keeps growing at the same pace, the amount of compute available 5 years from now will still be more than 10,000 times larger than today.
Of course, models will also grow bigger and require more compute other things being equal, but even if capabilities didn't improve (which they obviously will), it's hard to imagine that people wouldn't find useful things to do for agents to use all that compute.
I think it will take some time for people to figure out how to use them, but I have no doubt they will eventually.
Russia and Ukraine are competing for dwindling supplies of Chinese optical fibre used to tether jam-proof drones, helping to drive prices up more than sixfold as Moscow uses its greater resources to secure supplies. via @charles_clover@WillLangley96@FT
https://t.co/B4Uir0Osfh
a...notable exchange
Tim Miller asks if Europe is considering military escalation against Russia
Applebaum says leaders are discussing it privately but the public isn’t “ready” (yet!)
“Unfortunately...there'd have to be an event that convinced people it was really necessary”
The Lean FRO and Mathlib Initiative have released a joint statement on the Lean verification of the Riemann Hypothesis.
On November 20, Levent Alpöge released a proof of the Riemann Hypothesis as a Lean library of 17 million lines of code. Over the past day, the Lean and Mathlib communities worked together to verify it. The statement certifies the proof as correct and describes how it meets all necessary verification criteria.
Congratulations to Levent on this achievement, and thank you to everyone in the community who contributed to the verification.
Full statement: https://t.co/edxEbkodw7
Thank you Philippe Lemoine @phl43 for talking to me on this episode of Devil's Advocate about why the national security establishment is determined to undermine Jensen Huang; why Huang's misconduct is the main contributor to his downfall; why US export control on advanced semiconductors is bad policy; and why China's economic rise does not harm, but rather benefit the West.
https://t.co/ko0O7ckaXr
By the way, I’m well aware that the idea that computers could make central planning work (which again is explicitly *not* what I’m saying) is old, but the arguments people were making to that effect had specifically to do with how they could supposedly solve the calculation problem, whereas the point I’m making, which again is much weaker and leaves the biggest part of the calculation problem intact, is that because today’s AI agents have the capabilities to perform management tasks but have no interest of their own, they could solve many of the principal-agent problems that arise in the central planning from the fact that the interests of managers are not aligned with those of planners and they benefit from information asymmetries.
By the way, it's really interesting to read this in the age of AI, because I think you could make the case that relying on agents to manage enterprises and elaborate the plan could solve some of the principal-agent problems that plague central planning.
Of course, this wouldn't solve the fundamental problem with planning (which is the information problem Hayek talked about), but it's interesting to consider the possibility that, had AI existed back then, it might have helped make the Soviet economy somewhat less inefficient.
In The Socialist System, his famous book on the political economy of socialism (which I finally started reading after putting it off for years), Janos Kornai made the argument that the distribution principle of a communist society can never be applied because technological progress constantly creates new needs and, in the process, kind of anticipated the smartphone.
Not only have I not said that computers could save central planning, but I explicitly said it couldn’t. What I did say is that today’s AI could solve some of the principal-agent problems that arise in the planning process, which is 1) a much weaker claim, 2) one that is extremely plausible given the capabilities of today’s AI and 3) one that is obviously false of something like Cybersyn, which again is completely irrelevant anyway because Chile didn’t have a centrally planned economy under Allende in the first place, even if you insist on calling it AI to be cute.