For a more detailed summary of these results, check out our Substack post: https://t.co/td0qukAMbc
To explore more forecasts, including further questions from Wave 8, read the full report: https://t.co/SXxfvpwNkI
How are experts and superforecasters updating their expectations of AI progress?
In our latest round of Longitudinal Expert AI Panel (LEAP) forecasts, we asked for predictions on AI progress timelines, including updated forecasts for a question we last asked nine months ago.
We also asked for forecasts on near-term progress on @METR_Evals' time horizon benchmark, AGI timelines, the blockers and enablers of AGI, and AI's overall impact in the next 20 years.
🧵 Read on for more:
👍 Experts and superforecasters tend to view AI more optimistically than the general public
When asked for their view on AI's overall impact on the U.S. over the next 20 years, 57.5% of experts and 69.8% of superforecasters predict a somewhat or very positive impact, compared to 42.0% of the public.
The divergence is sharpest on problem-solving: 72.7% of experts expect AI will make people better at this, compared to 48.6% of the public.
One area in which experts and the public converge is their views on forming meaningful relationships. 68.4% of experts and 66.7% of the public expect AI to make people worse at this.
16.5% of experts and 11.3% of superforecasters expect AI's overall impact to be somewhat or very negative, compared to 29.8% of public respondents.
On a subset of ForecastBench questions, an LLM has matched superforecaster performance for the first time.
A submission from @GoogleDeepMind, named “green tree,” is now #1 on dataset questions on ForecastBench, our AI forecasting benchmark.
Superforecasters remain #1 overall.
Does parity on dataset questions mean ForecastBench is “solved”? No.
On market questions that require judgment about novel, one-off events, LLMs are still behind.
Even on dataset questions, the 64.9% achieved by Google DeepMind is unlikely to be the ceiling of what’s possible. ForecastBench doesn’t stop being useful at human parity. It will continue tracking LLM progress even if they surpass expert human forecasters.
To explore more Wave 7 forecasts from LEAP, including question-by-question details, visit: https://t.co/itWW3HWkLh
Find out more about LEAP, our panelists, and analysis from early waves on the LEAP website: https://t.co/VpXGhClZcM
To stay up-to-date with FRI's work, subscribe to our Substack: https://t.co/jUAJ1pc03u
How fast do experts and superforecasters expect progress in robotics to be?
In Wave 7 of the Longitudinal Expert AI Panel (LEAP) we asked forecasters to predict when robots will pass the "Coffee Test", and when they will perform the first autonomous appendectomy.
We also asked for forecasts on the supply of industrial robots, future U.S. labor share, and Amazon's e-commerce sales per distribution employee, to try to understand how fast one of the U.S.’s largest employers will automate.
🧵 Read on for more:
💲Experts forecast Amazon's e-commerce sales per distribution employee to rise
The median expert forecasts Amazon's real e-commerce net sales per distribution employee to rise from its 2024 baseline of $464,771.
They expect:
2026: $480,000
2030: $549,842
2040: $800,000
Superforecasters are even more bullish, predicting over $1 million by 2040.
Forecasters cited the suitability of warehouses for robotic deployment, Amazon's track-record in automation and a decoupling of sales from labor as key factors driving their predictions.