If Jevon’s holds, it’s bullish memory. The balance shifts from memory optimization to what more can we do with it.
For the last while the philosophy has been to optimize and do more with less memory and now, we expand the surface area such that experimentation will not have to consider this limitation.
Another piece worth noting is this. In prior cycles of memory demand/supply we have not had as much control from the teams to increase the value per bit as much as we have today.
This is evidenced by SK Hynix expanding value per bit with HBM, SanDisk expanding the value per bit on SLC NAND, and eventually HBF.
There are far more levers operationally from management teams to uphold the revenues and margins.
This month is a reminder to AI investors that the market price is not determined by results and tangible direction of the companies.
It is priced by expectations.
These expand and contract with innovation and disruption.
@HiCagr If hybrid linear attention makes 1M-context decode 6.3x cheaper, then workloads that were previously uneconomic become the default: hours-long autonomous agents, whole-codebase and whole context, always-on reasoning.
I think people are missing @bubbleboi’s why.
Think this will explain it…
The honest framing is that K3 is bearish specifically for the bandwidth-per-token and long-context KV legs of the memory thesis which are the legs people had been extrapolating most aggressively.
The original line of thinking was that as models grow we need a linear amount of memory with them (model size + context window), and it is correct that we were undersupplied on that trajectory.
K3 is a data point that it doesn’t.
It comes down to Kimi Hybrid Linear Attention (KDA). It lets you collapse all your history into a fixed sized state, thus increases to the context window don’t result in a linear scaling to the memory necessary.
The counter to this is Jevons Paradox which has until now held true.
Bingo.
But memory is the better expression of this outcome.
- no ties to GPUs/ASICs (inference innovation)
- memory is the oil of the AI cycle
Memory = the brain. Compute = food to fuel neutron activity.
Every AI panic cycle ends the same way: the headlines change, the GPUs don’t.
Chinese model. American model. Doesn’t matter. They all need roughly the same amount of compute to serve users. $AMZN
HFT firms are paying $100k a month for API access to get faster Trump post insights.
I’m paying his copywriter intern $10k a month and positioning before they even go out.
Levels to this game.
BREAKING: Trump Media is charging up to $100,000 a month for banks and trading firms to get faster millisecond access to Trump's Truth Social posts through the "Truth API" low-latency feeds before anyone else does, per FT.
This is directly aimed at creating a new revenue stream for Trump.
One element of the strange climate now is the scale and inaccessibility of data centers versus PCs. But post-memory-disruption, PCs will have 4TB+ of Flash storage with ultra high bandwidth and we'll just run these models locally and see they're no big deal.
Views from the Mag7.
Upside case: build compute, train models, distribute to customers using your product.
Downside case: sell your compute like AWS and profit if you fail to accomplish the original goal.
Smells good for capex acceleration then and the debt markets around them