Siv

who's winning the race to the bottom?
—
memory, semis/AI infra, macro

[Siv @lecun_]

I updated Codex this morning and watched the icon transform into ChatGPT in my dock.

The new consolidated app has two modes: Work and Codex, and plain chat is now a sidebar item.

The agent ate the chatbot and kept its name.

---

The subtler story in yesterday's OpenAI announcement was that they retired Atlas - 9 months after launch.

A different post-mortem than Sora, which died of compute economics. Feels like Atlas died from a lack of adoption, as it's still faster to do it yourself than to have an agent browse. But a browser is cheap to keep alive; they could've parked Atlas until agents got better.

Retiring it anyway is more of a verdict on the form factor - once agents do the navigating, do you even need a standalone browser?

Seems like agents will get a cloud browser on OpenAI's servers, but without a traditional browser overlay (no tabs, no URL bar, etc.). Humans for now can still use the browser embedded in the consolidated app, and for those who won't switch, keep the Chrome extension.

What happens to the ad market once agents do the clicking in a headless / no-browser way?

[Siv @lecun_]

@kimmonismus Lol agent loops are really ripping this thing. 414B prompt tokens/day vs <3B completion. But 2.2% tool call errors under that load is genuinely solid.

Chinese models probably cross 50% of token volume by year end (vs. 2% a year ago...)

[Siv @lecun_]

Then tell me where you disagree with this simple model and I'll subscribe.

1. H100 1-year rates rise 40% early this year with renewals printing at rates they were signed at 3 years ago (with some renewals locked until 2028)

2. Rubin ships later this year with bottlenecks gating how fast they roll out. E.g., of the 16GW announced to open this year, only 5GW is under construction

3. Blackwell deploys at scale through 2027 as the gigawatt campuses complete and power comes online, and Hopper renewal rates fall. By then, the 2023 vintage chips have earned for 4-5 years, which makes the longer depreciation schedules accurate.

4. Hoppers start to glut while the newest chips likely stay scarce. The writedown risk sits with H100s/H200s deployed in 2024-25 into $2-3/hr rents, which need ~3+ years to pay back, so their breakeven lands in 2028-29 right as the glut arrives. The 2023 fleet rented at $4-5/hr and paid itself back within its first contracts.

Sure neocloud credit stress can show up around the 27-28 renewals if rents reset lower as debt comes due. But probably only if leverage keeps growing, because the initial take-or-pay contracts are sized to pay back the hardware and retire the debt. So weak renewals are more of a margin problem, not a solvency one.

But where and how does Nvidia get hurt here?

[Siv @lecun_]

Intrigued how the full article would reconcile agreeing with Intrator's physical bottleneck while betting Hoppers quickly go obsolete - but I can't afford £37/mo for permabear content.

The depreciation story on the hypers caught my attention when you raised it last year but it's exhausted and priced in now, and a rounding error. The cash conversion cycle extending isn't a short-Nvidia thesis either. Tbh it's more bullish that they're able to tighten their chokehold on the supply chain.

Never liked the neocloud model - there's more interesting parts of the stack than a levered spread business. But you lose me when the trade becomes shorting the whole cycle. The only shortable thing is valuations being stretched in some corners of the cycle, not whether demand is real. And you're picking the wrong names.

[Siv @lecun_]

@sudeepXD19 @t_blom 5G sucks in London. It’s a lost cause underground and indoors, and outdoors congests to sub-30 mbps… and it’s the best covered city in the UK

[Siv @lecun_]

@kermankohli I think they hold all year and Trump changes his tune. Knowing what the Fed does early is worth a lot more to him.

[Siv @lecun_]

@kermankohli Andreessen is friendly with trump and advises the administration. Starting to make sense how Warsh got the job.

[Siv @lecun_]

@jimcramer ready to roll 5% off session highs. inverse cramer never sleeps

[Siv @lecun_]

My gf sent me a pic of a Mercedes CLA running NVIDIA's DRIVE system with a camera rig on the roof collecting street data in London. The system goes on sale in Germany in Q4 and rolls out across Europe from there.

$NVDA sold off today while all semis rallied on the leaked memo about $META building its own chip. It enters production in September, yet they want 14GW of compute by 2027. Most of that 14GW will run on Nvidia...

But more importantly, why is Nvidia still trading like a stale GPU story for Meta's workload? The Iris chip is for recommendation models running its feeds and ads. Was it ever in doubt that Meta would move its most repetitive in-house inference workload onto custom silicon? It also announced its agentic model, which was trained on Nvidia...

Custom silicon works when the workload is frozen and you'll never be able to design that for architectures evolving as fast as physical AI. Driving and robot models went from modular pipelines to end-to-end to reasoning models in just a couple years, and that churn likely won't settle before 2030. A custom chip takes three years to build and freezes a design for a target that will constantly evolve. Remember Dojo... Even Tesla who's the furthest ahead in physical AI needs to train on Nvidia cause no one can touch its flexibility. The market is also underestimating how hard it is to pull off a Google TPU which took a decade of hyperscaler balance sheet money. These custom chip programs are multi-year bets that only make sense when capital is cheap. If and when capital tightens, speculative silicon projects will get cut first and spend will go back to be concentrated on their actual business.

And all of that is still just the chip layer. Nvidia's deepest moat here might be the simulation and data loop. To train general purpose robots you need millions of hours of the machine's own action-outcome experience, and that data will only exist after the robots are deployed. So the only scalable source in the meantime is synthetically manufacturing that experience, and Nvidia is years ahead here. The models are open, and slowly but surely every physical AI company will be built on Nvidia's world similar to how this generation of AI startups got built on OpenAI and Anthropic.

I'm not sure if the market is uninterested in the story or just sleeping on it. And don't get me wrong, I'm long Broadcom and custom silicon but people are doing Nvidia way too dirty. Maybe the market starts caring once it gets a taste of automotive. One model that generalizes to any street instead of mapping city by city which took Waymo 11 years. Nvidia's system was announced this year and goes commercial the same year 😂

Maybe then the market pulls forward that every other machine will get built on the same stack. A stack that Nvidia trains and simulates on hardware at cost...

PS. Fittingly, the person my gf was chatting to in the coffee shop was a data privacy consultant. She walked over and shouted at the tester that her data isn't free.

[Siv @lecun_]

I don't think they're really planning on trying or going for AWS type enterprise cloud (they shut down Workplace). They'll probably look to do more colo than cloud, where they retain the option of wholesaling blocks of compute to labs - like what Elon is doing with Anthropic. I also don't think owning the infra is much of a hedge in itself. GPUs depreciate, and renting raw compute is a capital-heavy, lower-multiple business than anything Meta does.

[Siv @lecun_]

MS's chief equity strategist went on Bloomberg today basically describing the air pocket we're discussing. The trigger seems to be last week's report on $META renting out its capacity. The view that the spender/beneficiary divergence needs to normalize seems to be moving toward consensus.

If anything, to me, that lowers the odds of a steeper correction if the caution is becoming oversubscribed and is already getting de-risked.

Quick aside on that Meta news, because I think it was totally misread. Floating a compute rental business feels more like a free pump than a confession that they over-ordered. Meta is the only one of the big four without a cloud business - the thing the other three get paid a multiple premium for. Costs nothing to announce, and it turns their criticized capex pile into optionality on a new revenue stream. It's also a perfect example of the reframe I'm broadly talking about below - a hyper starting to take back control of the narrative instead of letting its vendors write it. Meta also reportedly asked Google just recently for more capacity... Funny enough, the report hit a day after my post last week. Nothing about structural demand changed that day - but the narrative moved, and infra names were down 10-20%. That's how underpositioned parts of the market were and still are for a narrative change. A lot of names are still pricing in uninterrupted acceleration.

But the Bloomberg segment w/ MS actually changes my near-term read a bit, just from how consensus this is starting to feel - and it's making me more constructive on memory short term. I originally expected a longer consolidation here, but last week's flush is starting to look like it could have been the capitulation, or close to it. But if the market's choice today is between a steeper correction and not handing dip buyers a clean entry, I think the setup favors the latter. $NVDA has felt like it needs a real flush before going again which feels like a loose end in the setup.

Regarding your comments. I fully agree hypers don't stop spending just cause of short-term FCF/stock pressure. But things feel different in this cycle compared to prior ones in how much less control it feels like they have over the spend loop. Also the hardware in this cycle IS the product and the narrative has been whoever owns the infra wins the race... The big four guided ~$725B of capex for 2026 which is about ~90%+ of operating cash flow. Amazon's FCF is projected to go negative this year on the build. Last quarter, Meta raised its guide citing memory prices and the stock fell ~9% -which I doubt is unrelated to them floating the rental business. I'm sure when some of these guides were set last year, they thought this was going to look like more of a sprint. Now I think they're more susceptible to the idea (given what their share price is telling them) that this might be more of a marathon.

That's the psychological element entering the cycle for them. In dotcom, $AMZN could look at its share price and know the market was wrong. Here, if the market keeps pushing back on the velocity of AI capex, do the hypers start asking whether the market has part of it right - not on the structural demand, but on who captures the lion's share of the economics terminally? And the more structural the demand becomes, the more Jevons goes to work, the more obviously this is a marathon and not a sprint. And the longer it runs, the more sustainable supplier economics matter. So the real question becomes how much control are they willing to lose in the process of getting the capacity they need?

Another reason this cycle feels different is that they're not only buying for current or even modeled usage. They're buying for competitive survival, customer retention, supplier signaling - and what's effectively a very expensive call option on not being underbuilt when the next wave hits, with a matter of urgency they haven't had before.

My feeling that the narrative was due for a switch though came from consensus oversolving for chips over the hypers and losing sight of the actual power dynamics, and of the cards the hypers haven't played yet. And if this narrative change does catch wind, it becomes even harder for them not to act on it.

No one can actually fill in for their spend. It's not just the capital and the balance sheets - it's the customer and distribution layer and everything attached to it that can't be replicated with capital alone. You already see enterprises having trust issues with handing over data and workflows to anyone; those relationships are sticky, and not everybody would move their most critical infrastructure decision of the decade to a neocloud to cheap out. Also VC/PE capital can't survive harsh ROI periods the way a hyperscaler balance sheet can, and there isn't enough risk-taking capital at this point in the cycle to replace hypers spend anyway. Elon's the exception, and only because a permanent market premium lets him fund turning his company into one.

But hypers being irreplaceable is exactly what makes an air pocket possible. You saw airpockets following the enterprise buildout in the early-2010s and again with the demand pulled forward during covid. The demand was structural both times - and looking back, they were right - but it took roughly 18-24 months to digest, despite demand continuing to expand. Overextrapolating is in their nature, because the cost of underbuilding (lost customers/ground) always looks worse to them than the cost of overbuilding (an imperfect spend curve). This time is weird because the curve is just harder to define - which is exactly why the building is so much more aggressive. They're pushing 20%+ capex/revenue with Meta pushing past 50 vs running low-teens in prior cycles.

The digestion narrative is in the hypers' best interest. The capacity already committed through 27/28 should carry today's demand plus part of the next leg, so there's room to push back without breaking the buildout. Vendor margins need to start getting pressured, and the hypers need to rethink the economics they're ceding to the bottleneck suppliers as the spend starts to look more perpetual. Infra is currently being priced like the vendor knows someone else takes the capacity if one buyer pushes back - and I just don't think that accounts for the bargaining power hypers actually have. What makes it harder is that the cyclicals are much smarter about how past cycles hit them, and are managing supply to stay structurally short, or short enough. There is 100% serious game theory being run over those supply curves in the $MU/Hynix boardrooms - but same in the hypers', who also know a real pause could make suppliers slow their own investment and hurt their terminal COGS. Both sides are playing the long game against each other. But three years into this buildout and the vendors are clearly winning.

On the supply side, I think there's still a faulty consensus around the traditional econ view that high margins attract competition. That view has been laughable this entire buildout when you look at what leading edge tech looks like in EUV, CoWoS, HBM stacking, bonding, packaging, thermals. Forget the other parts of their moats around yield learning, customer quals, equipment/power access, or just sheer capital - the tech, patents, and R&D labs alone are insane.

So yes, in theory new entrants show up when margins are high - but in practice here, this is far from a supply chain that can be commoditized. So it does feel like if the economics are going to be pressured, it has to come from hypers flexing buyer power. It will not come from new supply flooding in.

And memory has an extra barrier on top of the one it shares with the rest of the leading edge, which is that as a historically brutally cyclical business, it would be nearly impossible to get IC approval to fund a new entrant or greenfield fab in memory. One of the biggest pieces of my memory thesis early last year was how much everyone's IC would overweight the downcycles.

It's the irony in memory's discounted multiple. Its old-cycle baggage is what continues to protect it from its biggest threat - new supply. It's kind of like shale after the 2014-20 overdrilling, except with much harder technical barriers on top. And the longer the market refuses to fund new competition, the stronger the moat gets and the more structurally dependent the AI stack becomes on the existing oligopoly. I've been calling it the scar tissue moat.

Look at the contrast with the rest of the stack. The higher-multiple bottlenecks are inviting competition precisely because they carry less baggage and a new-architecture premium. Elon was just approved for Mesh Optical, which does optical transceivers for data centers - and I think that's a direct result of photonics/optics trading close to 80x forward. He probably wants memory for the terafab too, knowing how critical it is, but my guess is it's a harder sell internally - memory would be dilutive to the $SPCX story in a way high-multiple optics isn't. Venture land is also funding more photonics, cooling, nuclear/fusion, quantum-adjacent compute - all the "newer architecture" bottlenecks - because public markets are rewarding stories with growth multiples memory still doesn't get. Counterintuitively this is making memory more structurally protected because its been "cheap".

The only credible new-entrant risk you can underwrite near term is China/sovereign-backed supply and CXMT, because they don't optimize for DCF math - they're building for national security/autonomy. And the early version of that thesis has played out so far in the obvious way where CXMT is absorbing real commodity share in domestic/server DRAM and NAND - but it's nowhere close to pressuring HBM. It's cost per bit still runs 30%+ above the big three, its DDR5 die is ~40% larger, and in HBM it's still sampling HBM2/3 vs HBM4. But honestly the flood risk has always been there and continues to be the main overhang on all this. I still think some of it is priced in though.

This is also what's most relevant to the "it's different this time" debate. The technical barriers I described always existed but nothing like today. From how much wafer HBM/DDR5 burns to the stacking/packaging steps that exist today. In past cycles, generic bits could actually meet the needs of the use cases but feeding todays accelerators built for longer context, video, agentic workflows, real-time inference is a different game.

The honest flip side to flag though is that overspecializing in HBM (which I don't think Micron is doing) gives up some flexibility to pivot back to commodity if high-end demand gaps. A lot of the capex doesn't convert back into commodity output. And while I'd still bet on the incumbents re-taking any market they wanted, it would also mean a pivot back would flood the commodity side. Food for thought...

There's a broader riff I have that underlies all of this, which is that with the advent of AI, everything feels like a race to the bottom. In the bits world - intelligence, ideas, code, prompts, workflows, even some IP diffuse almost instantly. The physical world is different so far. The velocity of intelligence is orders of magnitude faster than the velocity of atoms. Fabs still move on real-world timelines, and bots aren't disrupting that reality for a while. Which is why durable moats are collapsing toward distribution, truly unique data/feedback loops, and everything in the physical world - fabs, supply chains, customer quals, compliance, field engineering.

To zoom out on why I'm structurally maximalist on demand is that every prior compute cycle had a denominator. Ecommerce as a % of retail. Streaming, mobile, social, ads bounded by aggregate human attention. Cloud at least conceptually bounded by enterprise IT workloads. What is AI demand a % of? Software? Search? Labor? Knowledge work? Decisions? Curiosity? The upstream denominator is cognition itself and I wouldn't bet that has a visible cap. You'd be betting against the human desire to understand more, decide better, and act with more leverage. The enterprise ROI debate can feel narrow when there's a demand curve backed by pure curiosity running underneath everything - one that compounds whether or not the ROI gets proven on schedule. It won't absorb the 27/28 capacity, but it's a floor under the whole thing.

And as the cost of intelligence keeps falling through cheaper inference, better chips and model efficiency, the pie itself will expand (Jevons applied to cognition). New use cases will become economic, existing ones will scale in volume, and I think premium tiers continue to persist - where the highest-value work still pays up for better models, lower latency, better context, higher reliability, etc. I'm seeing many speculate that token costs go to zero and every model gets commoditized. But I think only the baseline commoditizes and the frontier keeps pricing power. It's a familiar economic structure.

The real limiting factor probably never becomes demand for intelligence but how much cognition humans/institutions are willing to operationalize (ie. trust, reliability, data access, permissions, regs, human in the loop requirements, and so on). I think these slow the curve but I'd hesitate to bet they cap it. Demand has been forcing its way through physical bottlenecks, and I'd expect it to do the same through trust constraints - with better controls and the right design of goals, incentives, and permissions. I take the safety and reliability concerns seriously, but so far I haven't seen a convincing case that they become a binding constraint on the demand curve rather than a friction that potentially slows it.

The constraint I take much more seriously today is data/context - and it matters for the more imminent air pocket conversation. Frontier progress so far has been framed around compute scale and architecture, with data mattering but never quite feeling like the binding constraint. But high-quality public internet text is pretty much fully mined out, so future capability legs need to come from higher signal data that is structurally harder to get because it's not something that is just scraped - it has to be captured, verified, and scored. E.g., outcome data (did the agent complete the task, did the KPI move), tool-use trajectories that teach models to operate rather than just answer, scientific/experimental data - maybe the scarcest category, because much of it has to be generated in the physical world first. And synthetic data doesn't get around this - it only works when grounded against something real, so the scarce input remains verification. Otherwise it's just recursive slop. And the friction extends further to the fact that some of this data already exists in bits but is locked inside enterprises and is private, proprietary, regulated - and some of it doesn't even exist yet and has to be generated in the physical world first.

So on the training side, we're all penciling in a continuous step-function of meaningful frontier progress, right as the inputs to that progress get structurally harder - and that's before counting the risk of being capacity-constrained to train the next models.

And on the inference side, context access is also needed - ie, a coding model wired into your repo or an enterprise model inside CRM/ERP. That access gates how fast we get to the real production workloads that absorb the outsized capacity being built.

Which connects to the recent Karp/enterprise-sovereignty point. A lot of my consultant friends onboarding AI inside big enterprises say their clients don't want to hand their data and internal decision logic to frontier labs - or even to Palantir for that matter. While it still confirms the demand, it exposes another risk for making the capacity path lumpier. Moving from easy frontier-API spend into private deployments, open-weight models, hybrid cloud/on-prem, or palantir-style permissioned control layers isn't instant. And plenty of workflows will land on smaller, specialized, more efficient models rather than the largest frontier ones. So I wouldn't count on enterprises "filling the gap" near term for that reason either. They'll become a bigger compute buyer over time, but there can be a timing gap between wanting AI and deploying it at scale - and that gap may also not be priced in.

On your self-funding question, I think there's no other option at this point. But based on run rates and backlogs, a sustainable spending loop looks like it can get going - and if you're long the hypers here, I think you benefit from that loop becoming visible. Also think you'll benefit from other surprise kickers like a consumer moment or agentic taking off.

Agreed on robotics/edge being a swing factor for 28/29. To be clear though, I do think agents and longer context can carry the demand if they take off - robotics/edge would be a great backstop if they don't scale to the level needed. If for whatever reason both slip, it does feel like another digestion catalyst - which would just mean the exponential pausing, not the acceleration writ large ending.

So net net, structurally bullish on the destination and the demand, but with digestions along the way - and the air-pocket case does seem to have more legs here. But hard to say whether it's been sufficiently de-risked. Will be interesting to follow the first hyper prints later this month (also stacked against a Fed meeting).

https://t.co/CfItDPFmCT

[Siv @lecun_]

Hypothetically if hyperscalers paused for too long others would eventually step in. But there’s little incentive for an extended pause. They’d only be motivated to potentially ease off in the near term without fully stepping away. I’m saying this in the context of their stocks facing pressure, but more importantly how much future demand is being priced into may infra plays (photonics, semicaps, etc.). Forward PEs look very expensive if growth slows even modestly.

That said, I’m less convinced others could meaningfully fill any real HS gap if a true air pocket materializes in 27/28. Sovereigns tend to copycat hyperscaler/frontier thinking rather than lead aggressively on their own. China will likely stay aggressive, but their more efficient models mean they won’t drive as much incremental compute demand on a relative basis. I wouldn’t count on neoclouds ramping up enough given their heavy debt loads from this cycle, and other private infra players are generally conservative by nature and won’t swing big to backfill. So while some substitution happens, I doubt they’ll be as aggressive as the HS have been.

This would open the door to a potential air pocket where caution starts outweighing fresh spend and proven demand. Again, this is just an asymmetric optic that isn’t priced in at all after this latest bull run - a possible near-term correction where the “indefinite demand > supply” narrative gets tested.

This is all speculative and is probably too early. The HS rhetoric has been nothing but aggressive so far. But any rally/relief valve in their stocks might be welcomed as a signal that they recognize the FCF strain and aren’t being reckless.

They’ve been able to afford this level of spending so far thanks to their balance sheets, and it has also helped motivate vendors to keep pushing supply. But for how much longer? Especially if they start needing to lever with debt before the next real demand wave hits?

The main missing piece of compute capacity right now seems tied to big agentic/agent workflows at enterprise scale. But even that might arrive slower than hoped (due to internal politics, red tape, other integration hurdles, learning curve). A lot of today’s visible demand is also hype driven/token maxxing - whether its enterprises testing things out or retail usage that may not translate into sustained high value workloads.

I do feel that memory isn’t in the same bucket as broader infra on a valuation basis. It’s hard to fathom we’re still in such good shape valuation-wise after this run. One thing I forgot to mention in my post is that it’s not only the multimodal needs and emerging use cases outpacing new efficiencies (around KV cache, quantization, etc.), but also the deeper architectural reality of the memory stack itself. Model weights/parameters and activations/intermediate tensors (and optimizer states/gradients for any training or fine-tuning) are often larger than KV cache needs in most workloads. And these layers can’t all be tackled with the same KV-specific compression or efficiency tricks.

[Siv @lecun_]

@WarMonitor3 Lol nobody thinks Iran is winning militarily. But that's irrelevant. Takes a few quid to bomb Ras Laffan, Abqaiq, drop some mines in Hormuz. Would be fucked for the global economy

[Siv @lecun_]

@HormuzLetter Literally sounds like a chatbot hallucinating

[Siv @lecun_]

@hardwarecanucks If @sama really ghosted samsung and sk hynix on stargate LOIs he'd be booted to the bottom of the allocation list which would be existential in this market

[Siv @lecun_]

@TheGalox_ @jukan05 New intel CPU packs 4TB max DDR5 RDIMM/MRDIMM - 2x its own last-gen W-3595X and 2x AMD's latest Threadripper Pro 9000WX

[Siv @lecun_]

To reach $30 EPS,
> Based on current production fabs in Taiwan (Taichung, Taoyuan), Japan (Hiroshima), and Singapore
> Plus capacity coming online through FY2029 (Idaho ID1 mid-2027, PSMC Tongluo 2H 2027, Singapore HBM packaging 2027, Singapore NAND Fab 10B 2H 2028)
> Assuming normalized gross/operating margins of ~40%/25% and full utilization
> Micron would need to generate ~$50B in annual revenue (less than current TTM)
> That would need to imply DRAM ASPs of ~$3–4/GB (down from ~$8-10 today) and enterprise NAND ASPs of ~$0.08–0.10/GB (down from ~$0.18–0.22 today) - a 55-65% ASP decline
> That would also need to assume no multi-year pricing agreements, no locked HBM ASPs, nor HBM's growing revenue base
> (And that's ALSO assuming HBM stays at ~3x the wafer area per GB vs. DDR5... It's already expanding to 4-5x with HBM4/4E. The idea that commodity bits can even be in oversupply by 2029 when HBM is consuming an ever-growing share of the same wafer capacity is very hard to reconcile)
> HBM today for Micron is generating ~$20–22B annualized. Even if you assumed that only reaches ~$25B by 2029 (against a "$100B TAM trajectory"), the remaining commodity DRAM and NAND would need to see ASP declines of 75%+, just to land at $50B
> That's deeper than any prior downcycle, which didn't have the structural demand floors that exist today (for AI training and inference data center workloads)
> Also can't assume any new modalities: no on-device inference, no AI PCs ramping to 32GB+ specs, no L3+ AVs scaling from ~16GB to 300GB+ of DRAM per car, no humanoids, no edge AI workstations
> You would need implausible bear conditions for ASPs to fall that much - and I'm not even fully confident we've even peaked yet

[Siv @lecun_]

Good breakdown but I'd push back on "structurally reduce DRAM demand" - really depends on what your expectations were going in. GPU and accelerator roadmaps are already building this in natively (ie. Blackwell ships with NVFP4 KV support).

Also the 6x is just on the KV cache. Inference DRAM is dominated by full-precision weights and activations. And training is obviously untouched too for those reasons (optimizer states alone). So the system-level reduction on a real workload is a fraction of that headline number - and what little it does move was already baked into HBM and DRAM demand forecasts.

The trajectory has been clear for the last 18+ months - this isn't new (KIVI, MLA, GQA, SnapKV). Wouldn't be surprised if 3-bit KV is already running in some labs and shipping in optimized kernels within months - but it's mostly a kernel story, architecture doesn't need to change.