In September, we described a few ways we're improving the accuracy and relevance of our AI progress forecasts. In the most recent wave from our Longitudinal Expert AI Panel (LEAP), we've experimented with a few potential improvements, including:
• Adding more questions with short-term resolutions to enable faster identification of more accurate forecasters
• Improving the information we give to forecasters as they're forecasting
• Adding new forecasters with track records of accuracy
• Separating out forecasts from a subgroup of experts who expect fast AI progress
• Repeating questions from earlier waves to understand how forecasters' views change over time
Read on for initial results from these experiments 🧵
We're improving the accuracy and relevance of the AI progress forecasts we publish.
Last week, we showed how the median expert, superforecaster, and member of the public in our studies have consistently underestimated AI capabilities progress (particularly on benchmarks) and have a mixed track record on other indicators.
Our findings suggest that median forecasts from these groups have not accurately predicted the trajectory of AI on some questions.
How might we get a better grasp of the trajectory of AI?
We're working on three major changes to the Longitudinal Expert AI Panel (LEAP) to address this question:
1. Sharing forecasts from a subsample of panelists who expect fast AI progress by 2040, for comparison
2. Adding regularly updated LLM forecasts to LEAP findings
3. Highlighting forecasts from the most accurate LEAP participants as soon as we have data to make this judgment rigorously
This thread is just a snapshot of some experiments we're trialing to improve the accuracy and relevance of LEAP forecasts. In this wave, we also gathered forecasts on:
• Congressional mentions of AGI risk
• AI spend per employee
• ECI scores of closed and open models
For more on Wave 13, read the Substack summarizing our results:
https://t.co/QIDKfaET83
To find out more about LEAP, visit our website: https://t.co/VpXGhCmx2k
Many thanks for the feedback, @finnhambly! We agree that the report over-claims a bit here. The crux framework is meant to capture the most commonly discussed pathway from automating AI R&D to accelerating capabilities. But AI R&D can certainly speed up with humans still in the loop. We note this briefly in Section 2.2: “full automation of AI R&D is not necessary for acceleration. In some standard growth models, a singularity (infinite output in finite time) can occur without full AI R&D automation”. We’ll make this more precise in v2 of the report!
Whether AI is about to undergo recursive self-improvement is one of the most important questions in the world right now.
I was surprised that we couldn't easily find a living literature review on the topic that covers relevant evidence and forecasts, so we (@Research_FRI) put together an initial version.
We review evidence including:
• Frontier companies' claims about how much of their AI research is currently being automated by AI
• Public benchmarks that help us understand AI R&D-relevant capabilities
• Forecasts from prediction platforms, prominent researchers, and AI models
Overall, the likelihood of major acceleration in AI capabilities soon seems high. For example, AI systems give a 35-56% chance of the rate of AI progress at least doubling by 2030, and, after reviewing the evidence, I'd personally put the probability substantially higher than that (>80%). In our report, we share a framework for assessing whether a doubling in AI progress will eventually lead to broader RSI-like acceleration.
Some reflections from our research on this topic below. 🧵
We are currently collecting forecasts from top human forecasters and AI experts. We plan to add them, along with evidence released after our data cutoff, to an updated version of the report.
Read the full report here: https://t.co/dorMslaMFQ
AI companies increasingly say they may soon reach recursive self-improvement (RSI).
With RSI, AI capability progress could accelerate dramatically. This acceleration could happen inside AI companies before outsiders can notice.
How close is RSI? Our new report reviews the evidence, turns key disagreements into forecasting questions, and gives initial LLM forecasts. 🧵
We note a few key limitations of our report:
• LLM forecasts: Several LLMs are now roughly as accurate as superforecasters on ForecastBench, our forecasting benchmark. However, we know less about their accuracy on complex questions with little base-rate information (such as questions about acceleration of AI progress). Also, we elicited the LLM forecasts before the new METR report was published.
• Data cutoff: Our data cutoff is Nov 3. The evidence in the field is moving rapidly, and so the report doesn’t fully cover, for example, the new METR report, Epoch’s Bayesian update to the ECI, or Claude Mythos/Fable 5.2.
We are currently collecting forecasts from top human forecasters and AI experts. We plan to add them, along with evidence released after our data cutoff, to an updated version of the report.
Read the full report here: https://t.co/dorMslaMFQ
We note a few key limitations of our report:
• LLM forecasts: Several LLMs are now roughly as accurate as superforecasters on ForecastBench, our forecasting benchmark. However, we know less about their accuracy on complex questions with little base-rate information (such as questions about acceleration of AI progress). Also, we elicited the LLM forecasts before the new METR report was published.
• Data cutoff: Our data cutoff is Nov 3. The evidence in the field is moving rapidly, and so the report doesn’t fully cover, for example, the new METR report, Epoch’s Bayesian update to the ECI, or Claude Mythos/Fable 5.2.