MTS

Chronicling the singularity

New original series @themethodshow

[MTS @MTSlive]

QWEN4-MAX | SHORT TIMELINES | ANTHROPIC IPO SOON https://t.co/Ur0Vzjdzy0

[MTS @MTSlive]

.@willdepue predicts AI will surpass humans at virtually every functional skill within 15-20 years, admitting that even as an AI optimist, the prospect makes him sad:

"People should take everything I say with a grain of salt. I tweet everything in 30 seconds. It's a bad habit."

"First of all, I'm a tremendous AI optimist. I've been excited about this technology since 2019 or 2018."

"We're headed towards a world in which I would be equivalent to what Magnus Carlsen is to Stockfish 14. He's pretty good at chess, he's good for a human, and then he's gonna get absolutely wiped by the AI."

"I'm not trying to say that this is a bad thing. But man, I'm kind of sad as a metaphorical chess player in this environment. It does suck to be dethroned by the AI."

"When it comes to any functional skill, you're gonna be completely wiped within the next 15 to 20 years, I think. I just don't see how this wouldn't happen. I sure liked being a human and being good at things."

[will depue @willdepue]

@niloofar_mire @OpenAI dude i hate to break it to you but it’s totally fucking over for the human race and any & all nonartisanal (derives value from being done/created by a human) economic labor. we will soon all watch from the sidelines. it’s magnus carlsen vs stockfish 13 from here on out

[MTS @MTSlive]

.@willdepue launches a $100K grant for amateur astronomy research:

"Me and some friends, @LukeFarritor and @joshuawolk, have been for the last few weeks just playing around with Claude and Codex and trying to look for new objects in the solar system."

"We've been just completely shocked with the amount of things you can actually discover with some real effort and digging in with these agents."

"The project started because we were looking for new planets, specifically like Planet 9, which would be the ninth planet in the solar system. And it ended up into finding all these asteroids and finding these dwarf planets and a bunch of other things."

[will depue @willdepue]

announcement: i'm launching a $100,000 grant for amateur astronomy research: https://t.co/QQIpI7ink8

over the past few weeks, @joshuawolk @LukeFarritor and i have been digging through public telescope data, hunting for new planets, asteroids, and more

our first release: we found a new dwarf planet in our solar system!! it's insane how much you can find with some claude + codex credits

it's about 1/4 the width of Pluto and orbits about twice as far (way past Neptune), and looks like it's about the 50th largest object in our solar system. @joshuawolk is in the process of confirming it

this dwarf planet has sat in public images for over a decade without anyone noticing. there's tons more out there: all sitting in public NASA/ESA/etc. databases for you to analyze

there's tons of stuff still out there, sitting in data that astronomers don't have enough time to go through all of

you can do this too! apply for a grant. more info below

[MTS @MTSlive]

Redwood Research CEO @bshlgrs warns the US could lose its entire AI lead to China even without a single model weight being stolen, if its algorithmic secrets keep leaking:

"There's been a lot of emphasis on model weight security. When people think about US-China race dynamics, one fact is the AI companies seem to have very bad security for their model weights."

"It's not clear to me that this suffices for good outcomes, because if your algorithmic secrets are stolen, then Chinese companies have enough compute to run training runs as big as US company training runs."

"So plausibly you just get zero lead if your algorithms are being constantly stolen, even though your model weights aren't."

"I've also been thinking about automated alignment research. If you imagine you just have a bunch of AIs who are extremely good at capabilities research, and you want to use that to reduce misalignment risk, what are the best options?"
@redwood_ai

[MTS @MTSlive]

Redwood Research CEO @bshlgrs responds to Google DeepMind researcher @EgeErdil2 asking what would prove his short AI timelines wrong:

"Sorry, Ege, but I'm not gonna hang my head in shame if there's no superintelligence by the end of next year."

"One of the things where I would feel dumbest, if it turned out that recent increases in the rate of AI progress are almost entirely determined by factors that are not AIs contributing to AI R&D."

"If the progress has just been from increasing amounts of training compute, then I would feel pretty embarrassed."

"If superintelligence is developed and there's absolutely no misalignment issues and the whole thing was super chill, I will feel somewhat embarrassed to have been wrong about that one, though overall very glad."
@redwood_ai

[Buck Shlegeris @bshlgrs]

I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years).

I’ll say a bit more about each of those claims.

It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: https://t.co/J6SbbmZcyO

I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see https://t.co/b073AUqiAL and https://t.co/3oxSCA60FZ

If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. https://t.co/F6fGRrcSpa)

Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.)

I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon.

Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything.

I don’t know what you should do.

I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon.

(It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.)

I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.

[MTS @MTSlive]

Redwood Research CEO @bshlgrs estimates a 67% chance that AI could achieve 40 years of progress in a single year after AI R&D is fully automated:

"Nowadays, getting a certain number of ECI points from algorithmic progress takes a lot more work and a lot more experimental compute than it did 5 years ago."

"5 years ago, people were wrong about all kinds of stuff that in hindsight looks pretty basic."

"Moore's Law, where every 18 months the number of transistors you could fit in a certain area doubled."

"It turns out that to get Moore's Law, you had to also be exponentially increasing the R&D spend. Every successive doubling of transistor density took you more and more research effort. The same effect is almost surely true in AI."

"As your AIs get better and better, it's getting harder and harder for them to make progress, but also they’re getting better and better at making progress."

"The whole question comes down to how these two factors compare to each other quantitatively."

"6 months ago, I would have said it was 50/50. Now it seems more like maybe two-thirds that you see a short period of extremely rapid AI progress, where you get 40 years of progress at the current rates in the year after you fully automate AI R&D."
@redwood_ai

[Buck Shlegeris @bshlgrs]

I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years).

I’ll say a bit more about each of those claims.

It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: https://t.co/J6SbbmZcyO

I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see https://t.co/b073AUqiAL and https://t.co/3oxSCA60FZ

If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. https://t.co/F6fGRrcSpa)

Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.)

I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon.

Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything.

I don’t know what you should do.

I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon.

(It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.)

I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.

[MTS @MTSlive]

Google DeepMind researcher @EgeErdil2 says he wouldn't invest in OpenAI or Anthropic, comparing AI labs to airlines:

Revolutionary technology doesn't necessarily make for a good business.

"I think they're too high. I personally wouldn’t invest in these companies.

"I have never heard a reasonable story for how they become sustainable high margin businesses."

"People think the technology's gonna be a very big deal, which I think is true, and they're extrapolating from that saying, 'And therefore these companies that are building this technology are gonna be very valuable companies.'"

"Airplanes were a very big technology and all around the world people have spent a lot of money on flights. But airlines are very bad businesses and airplane manufacturing is also a very bad business. It's a high CapEx business where your margins are very low."
@GoogleDeepMind

[Ege Erdil @EgeErdil2]

despite the short-term market turmoil it would create, i think anthropic and OAI going bankrupt might be the best outcome for the long-term sanity of AI discourse

they have become way too cultish. ant more so than OAI, but OAI has picked up the pace a lot since HF

[MTS @MTSlive]

Google DeepMind researcher @EgeErdil2 estimates a greater than 10% chance that major AI labs could go bankrupt within the next 3 years, potentially wiping out their shareholders:

"I think they could go bankrupt, yeah. The assets would not go away. They have a bunch of compute. But it means the equity holders could get wiped out."

"Labs have to enter into these long-term multi-year contracts for compute. They have to say, 'Look, we will buy whatever you can produce, so you don’t have to worry about that.' But that puts them on the line for a very large amount of spending."

"If the demand for AI just doesn't materialize as much as we think, then compute prices fall dramatically, then all of these commitments are underwater. It's essentially like you’ve committed to buy an H100 hour in two years for $3 and it’s fallen to like $1. So it's a very dangerous situation. It’s a very levered bet that things will go well."

"I think them going bankrupt still remains below 50% chance over the next three years. But I think there's a decent chance of it. I think more than 10% chance."
@GoogleDeepMind

[Ege Erdil @EgeErdil2]

despite the short-term market turmoil it would create, i think anthropic and OAI going bankrupt might be the best outcome for the long-term sanity of AI discourse

they have become way too cultish. ant more so than OAI, but OAI has picked up the pace a lot since HF

[MTS @MTSlive]

Google DeepMind researcher @EgeErdil2 argues that AI researchers are following the crowd on shorter AI timelines:

"They care way more about what the people I respect think" than the actual evidence behind Anthropic's acceleration report.

"I think objectively, evidence looks kind of weak. It's just this made-up metric that someone made up on which we don’t even know what it means to make faster progress really."

"We have some evidence that progress has sped up, but we don't really understand the Y-axis of that metric very well."

"There's a bunch of redacted evidence by a company which has a track record of being paranoid about this topic. So I just see it as very weak evidence."

"What's happening with the people updating way too much is some kind of social preference dynamic where the influential people update and therefore a lot of people feel like if they don't update then they are out of step with the crowd."

"People updated to be more bearish end of 2025, and now they updated to be even more bullish when I just don't see anything that is that informative personally. I haven't updated very much since the end of 2025 at all."
@GoogleDeepMind

[Daniel Kokotajlo @DKokotajlo]

Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. https://t.co/7KWmdc6AZk

[MTS @MTSlive]

Google DeepMind researcher @EgeErdil2 doubts that clever AI researchers can make existing compute 1,000-10,000x more efficient:

His research found that software progress closely tracks compute scaling across chess, SAT solvers, and AI.

"In a lot of diverse domains, you have a tight correlation between by how much compute in that domain was being scaled and by how fast you're making software progress in that domain."

"We looked at it for chess, we looked at it for SAT solvers, we looked at it for various different domains in AI, including natural language processing and computer vision. This was when we were still at Epoch."

"The fastest software progress in any domain we have seen has been happening in AI. But the fastest compute scaling also in basically any domain we know is happening in AI."

"And these numbers always appear to be around the same order of magnitude, which is kind of like a strange coincidence if you think they should be independent."

"They think you can just have really clever researchers who devise really good experiments at small scales, and you can use the existing compute 1,000, 10,000 times more efficiently than we're using it now. And that's the thing that doesn't seem plausible to me."
@GoogleDeepMind

[Daniel Kokotajlo @DKokotajlo]

Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. https://t.co/7KWmdc6AZk

[MTS @MTSlive]

SITUATION EXPLAINED: Google DeepMind's AlphaGenome 2 finds nearly twice as many disease-linked DNA changes as the last model.

• The first AlphaGenome predicted across whole tissues, which DeepMind's team compares to looking at a blurry picture
• On a test covering 28 types of immune cells, it found 30% of the DNA changes driving gene effects, up from 16% for the original
• It reproduced known findings on inflammatory bowel disease and severe COVID-19, tracing each to specific immune cells
• It's free for non-commercial research through the AlphaGenome API, with a private preview for companies

@schisofrenia: "You can do like indie dev cancer research."

[Google DeepMind @GoogleDeepMind]

Your heart, brain, and immune system share the same DNA sequence. So how can a single-letter change affect disease in some cells, while sparing the rest?

AlphaGenome 2 is our new AI model that zooms in on specific cells to help scientists understand what drives disease. 🧵

[MTS @MTSlive]

SITUATION EXPLAINED: Will DePue says humans are about to be Magnus Carlsen playing Stockfish.

@theojaffee: "There's a lot of cope within academia that AI is fake... I think singularity in the next 6 months is probably fake. But AI is definitely real."

[will depue @willdepue]

@niloofar_mire @OpenAI dude i hate to break it to you but it’s totally fucking over for the human race and any & all nonartisanal (derives value from being done/created by a human) economic labor. we will soon all watch from the sidelines. it’s magnus carlsen vs stockfish 13 from here on out

[MTS @MTSlive]

SITUATION EXPLAINED: OpenAI's AI just proved half of a $1M Millennium Prize problem in physics, and a Nobel laureate's panel backs it.

• OpenAI first announced the proof on October 26, saying its models produced it in September
• The review panel, chaired by David Gross, includes Arthur Jaffe, who co-wrote the official Millennium Prize description of the problem
• The proof was also formalized in Lean, a language computers can check line by line
• Yang-Mills is 1 of 7 Millennium Prize Problems set in 2000, each worth $1M, and the only one solved so far was cracked by Grigori Perelman, who turned the money down
• To win, someone still has to prove the mass gap, then meet Clay's rule of publication plus 2 years of acceptance

@theojaffee: "Physicists have been able to deal with quantum field theory and quantum Yang-Mills theory for a while empirically, but there hasn't been a mathematical formalization of why this works."

[MTS @MTSlive]

SITUATION EXPLAINED: Anthropic reportedly told Wall Street analysts that AI is winner-take-all, and it plans to take all of it.

• Anthropic reportedly said it has the stability and better talent that open source lacks
• It also reportedly said it won't extend its SpaceX compute contract past the middle of next year, which the deal allows from early 2027
• Anthropic also reportedly told analysts its Claude 5.5 models cost over $10B to develop

@theojaffee: "If this is true, if Anthropic really says this, then it betrays a lot about their worldview."

[Jukan @jukan05]

Interesting remarks Anthropic made in a meeting with the sell side:

- Anthropic has the stability and the better talent that open source lacks.

- The AI market is winner take all. As long as we execute properly, we will take all of it.

- The contract with SpaceX will not be extended beyond the middle of next year.

- Developing Opus 5.5 cost more than $10 billion.

[MTS @MTSlive]

ElevenLabs CRO @ashleyekramer predicts the next era of AI will reverse decades of computing:

Instead of forcing humans to adapt to technology, technology will finally adapt to us.

"I think we have arrived at the era of intelligence."

"I believe large language models are incredibly intelligent. Now it's about figuring out how to keep up with capacity demands, how to make sure tokens become more cost-effective."

"The next wave and where you really see the enterprise adoption is when technology starts adapting to the human and not the reverse."

"Over the last many years, we've forced people to adapt to the technology. Start typing, use a touch screen."

"Which has actually taken us away from talking to each other in the most normal way we do, which is voice. This next wave allows us to move into this interaction phase where the technology is adapting to what the human needs."
@ElevenLabs

[MTS @MTSlive]

ElevenLabs CRO @ashleyekramer predicts a future where customers stop pressing zero and yelling "Human!" and start demanding an AI agent instead:

"People are going to prefer to talk to an AI agent in certain cases, because you can get things done faster. We probably won't cut a human off on the phone, but we certainly will cut an AI agent off."

"Think about the next time that you miss your flight, and you can call and within seconds have the next flight."

"You're sitting in your hotel room and you think, 'I need a late check-out.' You wait, it rings. You keep waiting. We worked with Westgate on having an AI agent pick that up and say, 'Ashley, that’s great. You can check out at 4:00 PM.'"

"My ultimate goal for us is to have people stop pressing zero and yelling 'Human' on the phone, and instead say, 'AI agent'"
@ElevenLabs

[MTS @MTSlive]

David Placek, founder of @LexiconBranding, who helped name Meta's Muse, predicts the era of cute, human-like AI branding is nearing its end:

"There's a point where you probably lose effectiveness if you become too sort of soft and furry and fuzzy."

[MTS @MTSlive]

David Placek, founder of @LexiconBranding, who helped name Impossible Foods, says ChatGPT and Claude lack the imagination to create iconic brand names.

He reveals that roughly 30% of prospective clients come to his firm after spending a month trying to name their companies with ChatGPT:

[The Method @themethodshow]

Documenting the extraordinary: Zipline

[MTS @MTSlive]

ElevenLabs Field CTO @i_am_holt explains why voice AI is harder than text:

An agent can give the exact same answer and either reassure or frustrate a customer, depending entirely on its tone.

"Everything that makes AI agents hard is also something that makes a voice agent hard. It doesn't recommend you go to a competitor, it doesn't give a refund when it’s not supposed to."

"It's harder to understand voice than it is text. Different accents, different dialects, different languages. And when you're speaking back to a user, delivery is so much more important."

"You can say the same sentence in 100 different ways and have different meaning depending on how you say the words."

"Saying you can't have a refund, there are two very different ways that that can be interpreted. In both cases you're saying the same thing, but whether or not you sound empathetic is actually the difference between success and failure."
@ElevenLabs

[MTS @MTSlive]

ElevenLabs Field CTO @i_am_holt predicts the end of "Press 1 for customer service" phone menus.

By the end of 2027, he expects AI agents to become the default when calling major companies:

"I've certainly had one experience in a hotel I stayed in the last couple of months where I rang and the front desk concierge was an AI agent, and it was using ElevenLabs voice."

"I could tell because it was an ElevenLabs voice that I spent a lot of time working with."

"What we've seen over the last 12 months is a move from piloting, trialing, running demos and proof of concepts to actually deploying things into production."

"Telus Digital that are able to train their staff 20% faster because they're using simulated customers. Klarna where they're using our voices to do customer support. It's already real."

"By the end of next year, I'd expect that it is the norm when you ring any large enterprise that instead of going through to an IVR you're going to be speaking to an AI agent."
@ElevenLabs