I've just drafted a new blogpost
"GPU demand is (~1Mx) distorted by efficiency problems which are being solved"
Mid-2024, Andrej Karpathy trained GPT-2 for $20. Six months later, Andreessen Horowitz reported LLM costs falling 10x annually. Two months after that, DeepSeek shocked markets with radical reductions in training and inference requirements.
For AI researchers, this is all good news. For executives, policymakers, and investors forecasting GPU demand... less so. Many were caught off guard.
The problem isn’t that executives / policymakers / investors lacked access to information per se… it’s that the technical/non-technical divide prevents them from seeing the difference between waste-based GPU demand and fundamental GPU demand.
Meanwhile, tech experts like Karpathy, a16z, and DeepSeek understand fundamental principles which are easy to overlook if you’re not implementing the algorithms yourself. But in presenting their results as merely “AI progress”, they buried the lede…
The Lede: If version X of an algorithm achieves the same result as version X-1 at 1/10th the compute cost, what exactly were we paying for in version X-1?
The answer has profound implications for anyone forecasting future GPU demand: version X-1 was roughly 90% waste. And a16z’s report, Karpathy’s achievement, and DeepSeek’s breakthrough indicate this isn’t a single 12-month event… it’s a multi-year pattern. Version X-1 was 90% waste. Version X-2 was 99% waste. Version X-3...
Wait… Leading AI labs allow waste?
The obvious question: if this waste exists at such scale, wouldn’t the labs building these systems have eliminated it already?
They are eliminating it. That’s what the 10x annual cost reduction represents. While hardware cost reduction accounts for some of the annual efficiency gain, software updates from AI labs constitute the vast majority… an ~86% efficiency gain annually. The puzzle isn’t whether labs are optimising… clearly they are. The puzzle is why so much waste existed to eliminate in the first place... and how much remains.
... (link on profile page)
@PitcherList Did we stop doing podcasts for the team by team SP breakdowns?
Understood the incredibly busy time and appreciate the strong output from @PitcherList — but this was a great format/vehicle and just want to adjust expectations.
The last team SP review podcasts promised more.
This paper in Management Science has been cited more than 6,000 times. Wall Street execs, top govt officials, and even a former U.S. Vice President have all referenced it. It’s fatally flawed, and the scholarly community refuses to do anything about it.
https://t.co/K34v0vpUPE
@agraybee Vacation time has monetary value. Common practice in corporations for positions requiring higher education is to vacation as part of the compensation package — and pay out the accrued, unused vacation time when an employee departs. So, actually that is how $ works.
@RationalMale@TheTinMenBlog 💯
Men know how to build communities for men.
Activists spent the last 30 years systematically dismantling men-only spaces.
The few communities that remain are frequently subject to ridicule and criticism among the popular media.
@asymmetricinfo There is a difference between Tax Rates and Tax Revenue. The reduction of rates can result in increased tax revenue, as corresponding volume increases as well secondary, tertiary effects.
@catgyoung @Timothy98537991@joshuasweitz This logical fallacy is exactly what leads to unsustainable spending and multi-trillions in debt.
Administrative cuts are feasible; Private funding sources are available.
That’s the 2nd logical fallacy — projecting a 1-1 result without accounting for response or innovation.
@FlyWithLEVEL Level FLT BCN-BOS April 8 -CANCELLED at the gate. It took HOURS to get back through passport security to the ticket area. LEVEL left our checked luggage unsecured/unattended at baggage claim. It’s been almost a week - zero communication! Claim submitted - still no response!
@mfariacastro @bethrites @EricTopol@Brabbott42 Screening rates generally LOWER - especially for the young. Huge drop during COVID never came back as high as anticipated. With improved precision on cancer risk factors — generic screenings w/o risk indications declined. Increased incidence is not due to screening.
@chrissyfarr It's almost as if health is a multifactorial outcome in a constantly changing environment with multiple intervention attempts, unobservable influences and zero scientific controls.
@RichWilsonP361@spreaker Great review - thank you. Check out Cam's monthly stats. Turned his season around in the last 2 months. Aug+Sep = 311/453/420. Long adjustment to level makes sense given age. Some reason for optimism.
@mikemayer22@oneshiningmets Making “good on paper” individual transactions is far easier than executing successful team construction to win through the season and be competitive in the postseason
@RichWilsonP361 High performance talent (eg top 0.1 %) in nearly every industry earn substantial $$ decades after they leave the job in one form or another
Joe Public doesn’t know that MLB players aren’t getting anything that’s “special” given their talent level in their field. Long overdue.
@RichWilsonP361 Non-profit healthcare c-suite ( >$1 billion in revenue) comp packages include SIRP (supplemental retirement plan) that enable big big $$ to be competitive with for-profits and not draw bad press attention. Very similar tactic.