Between this and The Altruists, I feel like Hollywood does not understand the Bay, does not want to understand the Bay, and will pump out generic 'nerds suck' movies regardless of what complex villainy and hubris and ideological, catastrophic recklessness they ostensibly depict
I think we now have to accept that the crazy AI people (I use the term affectionately) are, at least, directionally correct.
- There is at least a small chance that recursive self-improvement (RSI) in the next couple of years leads quickly to “superintelligence”, which in turn leads to extremely rapid transformative effects, including a possibility of total disaster.
- Even if, like me, you are very sceptical of the most aggressive forecasts, they are directionally correct. You can quibble with the timelines of RSI, with the precise meaning of “superintelligence”, and with the pace of transformation and diffusion, but the basic story is right.
Short of political interventions, we will build systems with capabilities superior to ours across all domains within the next couple of decades. The limitations of current systems look solvable, progress has been shockingly fast, and the resources being thrown at AI are enormous. This process will likely be greatly accelerated once AI systems outcompete us at designing better AI systems, and it will lead to a speed and scale of transformation, as well as novel catastrophic risks, that almost no mainstream elites or normies are pricing into their models of the world.
That, at any rate, is what I now believe with a high degree of confidence.
Yes! Note, however, that Robbins's argument has shifted from "the only profs who will survive are those who know more than AI" to "old fashioned high-quality instruction is essential," which can be supplied even by middling profs who don't know nearly as much as Claude - like me.
I hope this piece is a wake-up call to universities that the mission has changed. Optimizing for anything but rigorous education isn't going to cut it.
Really important conversations on the topic of AI happening within universities. I don't think this article captures it (a large focus is on one school with an enrollment of 1,000 and one professor applying for a 'religious exemption' drives the narrative. Even the Georgetown example is presented as students not interested, but the data show a less clear story: 40% cautious/negative v 25% positive hopeful v 35% neutral.)
I *required* AI use in my class last year, and spent half a class on AI workflows to advance health research, and I got extremely positive comments from my students -they had not been getting this training in other courses, and they wanted it. There are certainly anti-AI folks, but there are also the opposite. (one group is more vocal.)
@snackowska Seems like it's going to devour us, or not, regardless of whether I use it for a bunch of useful things. So that's what I'm gonna do -- while calling my rep and demanding sensible regulation.
The point of cellphone bans isn’t simply to raise test scores. It’s to bring students back to each other, back to their teachers, and back to the business of learning.
Achievement gains are welcome. Rebuilding school culture is the bigger prize.
@acaguy@rbnmckenna86 Detection is what is, but Pangram is *very* reliable for cut and paste AI outputs. It can be defeated through paraphrasing and "humanizing," but it has vanishingly few false positives -- for now.
In short there are two gigantic views and camps emerging as we head into 2027(thematically it’s already 2027 though, 2026 ended around August, it was a short year).
SF and DC intelligence circles believe we are coming towards the birth of a machine god that they can use to destroy China and dominate the world for eons to come.
Elite NYC + Global finance as well as the CPC views this as utter delusion, understand the physical world, and see the creaky limits of the current AI trade. They think we’re headed for a Capex bubble and that while LLMs are useful, the technology will take a generation to usefully diffuse.
Neither side will be completely right by the end of 2027,
The key question is who is more ACCURATE by the end of 2027.
2027 is going to be a movie of a year.
@snackowska@_akpiper@josebowen It can undermine the development of reading, writing, and critical thinking capacities in all kinds of ways. This is why AI should be a subject of critical analysis! Asking undergrads to explore its use in some disciplinary contexts probably make sense, too.
@snackowska You don't have to like AI (I'm conflicted about it) to see that this is not true. AI can be used in service of all kinds of advanced inquiry (surface reading at scale, pattern matching, etc). Check out the work of @_akpiper or @josebowen. Now, is it beneficial to most undergrads?
I’ve read this a couple times now, and while there is, frankly, a lot of sloppy argumentation (which I will get to), the charitable, simplified read of Dwarkesh's position seems to be:
1) AI needs major breakthroughs to achieve AGI/ASI (continual learning, sample efficiency, etc.) — this is a view he’s held for some time now, in line with Ilya
2) These breakthroughs, and the ML research skills required to obtain them are verifiable, and thus current, pre-breakthrough AIs can hill-climb on these sorts of problems and capabilities to solve them (like we saw in math)
3) Once these breakthroughs occur, we *may* see rapid, recursively compounding gains to intelligence, leading to ASI in a very short period of time
There's a ton of other stuff in the essay, but I think this is the basic crux. Christiano basically articulated this same argument in his Dwarkesh appearance, and if you look at AI-2027, it essentially follows the same trajectory: Agent 2/3 makes a ton of algorithmic progress, allowing Agent 4/5 to explode.
In general, I think the high-level narrative here is...fine? It’s certainly one plausible outcome. But, imo, it is a much narrower path than I think many are giving it credit for—a lot of things actually have to go just right for this to play out. And here's where I return to the other, messier arguments, because many of the intuitions and arguments driving Dwarkesh's view in favor of near-term ASI (e.g., EO2027) as a highly probability outcome (~50%) are, imo, quite bad:
- “[We'd see] a jump as big as GPT-3.5 to GPT-6 Astra in just 12 months” -> No, under this scenario we’ll see ~4 years of AI progress in 1 year. If there are diminishing returns, then this won’t be as extreme as the jump we saw from GPT-3 to GPT-6. It very well could be, but that would require innovation to overcome the diminishing returns we'd expect in the base case.
- “The other big piece of evidence for transfer is human evolution” -> Except if AI was following the sort of pattern we saw for human evolution, we should have seen better transfer long before AI started solving millennium problems. This spikiness has been one of the big surprises of AI!
- “Rather, my claim is that scaling up RL training will give AIs the abilities necessary to ramp quickly in new contexts…it’ll go through an evolutionary process that involves centuries of corporate competition to figure out the optimal organizational techniques.” -> These are in direct conflict! If ramping requires RL runs that simulate centuries of competition within “digital economy full of millions of other AIs” then that’s quite bearish for ramp speed (relatively) given the compute and wall-clock requirements!
- “I have a lot of 20-something-year-old friends whom I’ve watched accomplish incredible things and build world-leading companies and ventures in domains they entered into with no previous experience.” -> lol. rip leopold. But also, again, these are humans, with human sample efficiency. If AI gets there, sure. But how soon it gets there is the big crux.
- “Obviously, some pieces of feedback—about how your actions impacted the quarterly earnings, say—require months of serial time. But even granting that, we should expect these future AIs to be able to get to top human performance in almost all domains within months of being deployed.” -> …no that’s not how that works at all because one set of earnings isn't enough to robustly learn from and you can’t re-run the same real-world conditions.
- “[Sample efficiency & online learning are] AI’s equivalent of a Millennium Prize problem. And mathematics has already shown us what happens to such problems” -> Sure, but there’s a difference between “AI can solve a Millennium-level problem” and “AI can solve *this specific Millennium-level problem*” -- if the blocker is a specific problem, the timing spread becomes much wider.
- “My answer to this [experiment/wall clock/compute] objection is that each agent will still have access to a ton of compute. Frontier labs have been more than quadrupling their compute every single year.” -> But that’s what they’ve needed to do keep up with their current pace! Even if the labs maintain their current pace of compute growth (and like, will they even? capital markets really do not seem accommodating and Dwarkesh's roommate just lowered his compute projections), we'd still need a step change in pace above that!
And that's not even covering the biggest crux of all: whether the returns to intelligence continue to recursively compound faster than diminishing returns eat into them.
Now, are any of these objections fatal? No. Do they “disprove” hard RSI/an intelligence explosion? No, ofc not. Am I kind of nitpicking here? Yeah, 100%.
And to Dwarkesh's credit, he does anticipate and acknowledge a scenario (the “spark”) where these objections play out.
But the reason I've bothered to rant this long is that I see a lot of AI/RSI discourse follow this pattern: where 50% of the arguments are reasonable and grounded and appropriately hedged, and then the other 50% are these sorts of sloppy intuition pumps and what-ifs that include a ton of detail but a noticeable lack of rigor. And I can’t help but get the sense that it is the latter 50% that is really driving people’s predictions and expectations here, as it leads to sentences like this:
“In the spark scenario…it takes years to get to the same endpoint that the software-only singularity would have achieved in months: a superintelligence that is to us what we are to chimps.”
Note that Dwarkesh thinks these two outcomes (spark vs. software-only singularity) are equally likely. 50/50. I’m sorry, but that’s absolutely nuts. Even Christiano put the likelihood of ASI by in 2027 *absent any slowing down* at only 1 in 3, and I still think that's too bullish.
Like, yes, I fully anticipate that we will have automated AI researchers (or something close to them) in 2027, and the universe where we go from that to a software-only singularity is not entirely out of the question. But obtaining that universe relies on a confluence of various frictions, diminishing returns, and wall-clocks being overcome, a somewhat specific set of algorithmic innovations being made or obviated, and an assortment of other factors playing out just-so to enable that perfect storm. Even a fairly rapid exponential doesn't qualify when you're talking about full-blown ASI in 13 months. It may not be impossible, but it's certainly not a 50/50 outcome.
I basically agree with this, but "teach like it's permanently the 1990s" is not the way to define it or frame it. It's a losing argument, just like "teach like it's permanently the 1890s" was a losing argument over the course of the 20th century. There's no need to frame this in such a nostalgic way.
Instead, optimal instruction needs to be defined as "uses whatever tools and facilities are needed to keep the analytical burden inside of students' brains." Functionally, that means pivoting from digital back to paper, or to real objects (e.g. real spherical globes, which an AI can't auto-analyze, rather than digital maps). It means pivoting to in-class testing and having proper facilities for that.
It also means picking reading types where none of the important information about the reading is conveyed from AI-generated summaries. This actually reenshrines many forms of fiction, where an engaged student would certainly prefer to learn what happens next by actually reading the story than reading an AI-generated list of spoilers.
Furthermore this reenshrines examples of philosophy and theory where the author's writing style is somehow a core component of the overall argument, and so the "summary" cannot actually convey the argument. Spengler is like this. Many other philosophers are too (though certainly not all).
Great film is like this too of course. Imagine not watching 2001: A Space Odyssey and instead only reading an AI-generated list of plot points. That student would have no idea what the movie was like.
Everyone is fixated on academic honesty or prize contests as test cases for Pangram, and no one seems to consider people in my position, editors at small magazines. We get dozens of slush-pile submissions every week, lots of them are AI-generated, and if we didn't have a tool to easily sort out suspect submissions, we'd have to just stop taking cold submissions altogether, which would be terrible for emerging writers.
How can you not be excited for the week ahead? Which CEO is getting honeypotted next? What part of our infrastructure will fail? Maybe we’ll invade a country. Perhaps we’ll rename the Pacific Ocean. Anything is possible!
As someone who uses Claude regularly, I find this completely insane. Claude’s coding ability is so useful, and its writing is still so bad! Why would you do this!!!