Tomasz Gutowski

[Tomasz Gutowski @tomgutowski_ai]

The bigger problem is a fundamental methodological failure: equally predictive models produce radically different acceleration estimates, including one compatible with no acceleration.

Pushing “roughly 2×” as the headline anyway is indefensible. Caveats don’t justify privileging a number the analysis cannot distinguish from competing estimates.

Presenting these results as anything other than "inconclusive" is pure marketing, not an "analytical report."

[Tomasz Gutowski @tomgutowski_ai]

https://t.co/whQd4dPl6B

[Tomasz Gutowski @tomgutowski_ai]

“AI progress has doubled. Superintelligence in 2027. The world as we know it is coming to an end.” 🚀

METR published a report estimating that Anthropic was improving its models roughly twice as fast as before, with most of that acceleration likely driven by AI helping with research and development.

The authors acknowledged substantial uncertainty but - as could have been expected - the discussion on social media was often more confident, with predictions of an intelligence explosion, imminent superintelligence and the end of the world as we know it.

As a proper human bottleneck, I decided to complicate things and read the report in full.

I haven’t done any calculations of my own or proposed alternative scenarios. I’ve simply taken what the authors themselves report and compared it with what has been circulating on social media.

In this first of three articles, I examine the public benchmark analysis behind that “nearly twice as fast” claim.

You'll find a link in the first comment :)

[Tomasz Gutowski @tomgutowski_ai]

@noti40159711 @bigd0gmx @taw_moto @vipinxtwt You’re not definitely right, bro. You aren’t even presenting an argument :) How does solving some problems while making obvious mistakes on others establish that GPT is “unequivocally the best mathematician in the world”?

[Tomasz Gutowski @tomgutowski_ai]

@noti40159711 @bigd0gmx @taw_moto @vipinxtwt So, for now, neither you nor those people were right ;) What evidence makes GPT “the best mathematician in the world”? Do you have data the rest of us don’t?

[Tomasz Gutowski @tomgutowski_ai]

@noti40159711 @bigd0gmx @taw_moto @vipinxtwt So in what sense were you right, given that it hasn't happened yet?

[Tomasz Gutowski @tomgutowski_ai]

@Solglasogoh @bigd0gmx @noti40159711 @taw_moto @vipinxtwt so that hasn't happened yet

[Tomasz Gutowski @tomgutowski_ai]

@krasmanalderey @Thirumagal_12 @theobearman METR didn’t establish that either. Their estimate changes from 1.1× to 2.9× depending on assumptions, and key evidence is redacted. There’s nothing publicly verifiable here to ground a serious discussion. Just claims we’re supposed to take on faith. That’s religious nonsense.

[Tomasz Gutowski @tomgutowski_ai]

@Thirumagal_12 @theobearman Because the filing explicitly says this determination is based on expectations about future models, not observations of current ones. So, for serious people, talking about this is a bit embarrassing.

[Tomasz Gutowski @tomgutowski_ai]

@1techutopiapls @GoodbodyJeff @ramez @alexolegimas @sebkrier You made a prediction, declared yself correct, invented your opponent’s future reaction, then called him intellectually inferior for it. You’re arguing with your own fan fiction bro. No evidence, just absolute conviction and abuse for anyone who won’t believe. Worthless drivel.

[Tomasz Gutowski @tomgutowski_ai]

@stanjns @CKeruac Rozumiem Twoją intuicję, ale przy tak złożonej materii jak LLM-y rozciąganie sukcesów w jednym zadaniu na zbiór wszystkich możliwych problemów jest błędem logicznym. Zresztą te zero-daye przechodziły ludzką weryfikację.