This week, we took a closer look at Bitcoin’s already interesting setup — and what it would take to make it even more compelling.
Read more here 👉🏼 https://t.co/MTwoubZ30Q
This was indeed unprecedented:
30-year Treasury yields have never been up this much on a Fed decision while two-year yields fell this much, based on data going back to August 1987.
Merrill: The S&P 500 has gained around 110% since October 2022
Historical comparisons suggest the rally may still have room to run, as bull markets since 1949 have averaged around 192% returns over around 5.5 years... though they lifted this from @ryandetrick
I agree with your core point: from the perspective of the active decode path for a single sequence, Kimi's hybrid linear attention significantly reduces KV fetch overhead. Roughly 75% of the layers maintain only a fixed-size recurrent state, shifting the critical bottleneck toward high-bandwidth state updates and dense matrix multiplications. From that perspective, it has better long-term scalability than a traditional full-attention architecture.
That said, We don't think this is incompatible with DRAM and NAND continuing to play an important role.
First, Kimi is not entirely fixed-state. The remaining ~25% of MLA layers still maintain KV that grows linearly with context length. Looking purely at long-context KV capacity, Kimi effectively eliminates about 75% of the linear growth (which Kimi itself has highlighted as the maximum KV reduction), whereas DeepSeek's approach still scales linearly with context but compresses the total KV footprint to roughly 10% of the original at 1M context.
Second, let's talk about Kimi's "75% fixed-state." The state is only truly fixed-size if you have **a single sequence moving forward continuously while keeping only the latest state**. In a real serving system, however, you need prefix caching, recovery from intermediate positions, and agent branching. That means you cannot simply keep the latest recurrent state—you need to preserve multiple intermediate checkpoints. While each checkpoint is fixed in size, the number of checkpoints grows with context length, checkpoint granularity, and branching. As a result, Kimi reduces the HBM working set for an active request, but it does not necessarily reduce the overall DRAM cache pool by the same proportion.
This becomes even more relevant in an agent swarm setting. A single task may spawn multiple agents, tool calls, and reasoning branches. Shared prefixes can be reused, but once branches diverge, each branch maintains its own recurrent state, the remaining 25% KV cache, and intermediate checkpoints for future recovery and further branching. In other words, reducing the memory footprint of each individual agent may actually enable the system to run more agents concurrently, preserve more branches, and support longer execution traces. The total cache footprint therefore may not decrease—and could even increase.
We also need to consider the enormous capacity and cost gap between HBM, commodity DRAM, and NAND. HBM should hold the hot working set that is actively being computed and is most sensitive to bandwidth and latency. It is simply not economical to keep every agent session, prefix checkpoint, and historical cache resident in HBM. A much more practical architecture is memory tiering: hot state stays in HBM, warm checkpoints and KV reside in CPU DRAM, while cold prefixes, inactive sessions, and long-term agent memory are offloaded to SSD/NAND.
Finally, I think GPU utilization is another critical angle. Agentic workloads and long-context tasks spend a significant amount of time outside active decoding—waiting for tool calls, synchronizing with other agents, planning the next step, and so on. If every KV cache remains permanently resident in HBM, a large fraction of GPU memory ends up occupied by inactive sessions, leaving less room for new active requests and ultimately reducing GPU utilization.
The value of offloading is therefore not just reducing HBM cost—it is about freeing scarce HBM capacity so that GPUs remain dedicated to active computation. When a session becomes inactive, its KV cache, checkpoints, or recurrent state can be migrated to DRAM or SSD, and then brought back into HBM when execution resumes. This kind of memory tiering is conceptually similar to CPU cache hierarchies or operating system paging: the objective is maximizing cluster throughput rather than keeping all data in the fastest memory all the time.
Now consider Agent Swarms, where a single workload may involve dozens or even hundreds of concurrent agents, each with its own context, checkpoints, and memory. Even if Kimi substantially reduces the working set of each individual agent, the more likely outcome is that the system simply runs many more agents concurrently—not that all state permanently resides in HBM. In that sense, I view Kimi as changing the division of labor between HBM, DRAM, and SSD, rather than eliminating the need for DRAM or NAND. As long as there remains a large capacity and cost gap between HBM and DRAM/SSD, memory tiering and KV offloading will continue to be fundamental building blocks of large-scale inference systems.
Will be back to the US next week.
Talked to a few PMs during my trip in the last two weeks. I’d like to share some interesting findings:
Sentiment: Overall very panicked and confused.
Positioning:
1. Shanghai HFs are extremely long on memory names, partly due to the TRS restriction I mentioned a few weeks ago. They are quickly selling CN semis across the board and buying DRAM ETFs to lower margin pressure.
2. HK HFs are mixed and divided; bulls and bears are balanced.
3. US LOs are waiting—not about stock prices but still don’t believe LTAs can turn memory names from cyclical to growth valuations.
4. Macro HFs sold all of their memory/semi positions.
Interesting points from both sides:
Bulls:
1. Supply chain checks suggest prices and margins will remain higher for longer.
2. AI capex in 27/28 will grow much higher than most expected.
3. OAI/ATH’s ARR shows no signs of plateauing. Codex’s WAU is growing crazily right now.
4. Kimi K3’s tech report literally suggests every semi component is a bottleneck 😂
5. No macro risks; CPI/PPI much cooler.
6. Even if memory spot prices can’t go higher, these memory companies can do large buybacks or dividends to support the stock price.
7. OAI/ATH is hiring a large team of solution engineers to quickly build vertical solutions (finance/healthcare/legal) to find the next S-curve beyond coding.
8. Even if memory names fade, the entire semi sector will remain intact.
Bears:
1. Higher memory prices are not sustainable and will be demand destructive. Clients will push back on further price hikes, and even the government may step in to force massive production expansion.
2. Memory prices will peak in Q4.
3. No doubt on 27 capex, but 28 capex budgets are clouded.
4. Even if fundamentals are solid, the position is extremely crowded and Korean retailers’ leverage is a huge concern.
5. Even though CSPs’ capex planning is aggressive for 27, a large portion of deliveries will be delayed due to power supply constraints.
6. No intermediate story of fast AI application adoption like AI coding.
7. Higher prices and more LTAs during the upcycle can’t prove memory names aren’t cyclical anymore. We need to see resilience during the downcycle to prove that, like memory names keeping the lower bound of margins in a downturn.
8. Memory is the core of AI: if traded memory dies, then everything dies.
9. Memory has no difference vs. other commodities such as silver, oil, or lithium carbonate. When they peak, their narrative and fundamentals all look fucking good 😄
Codex has been eating your SSD
For weeks. Through a logger nobody opted into.
The bug: TRACE-level SQLite writes hammering ~/.codex/logs_2.sqlite around the clock. One dev measured 37 TB written in 21 days.
Annualized - 640 TB/year.
A typical 1 TB SSD is rated for 600 TBW. That's your entire warranted lifespan burned in under 12 months.
OpenAI shipped fixes in 0.142.x on June 22 - cut writes by ~85%. But the third PR only landed in 0.143.0 alpha, and r/codex is still filling up with the same reports weeks later.
One user: Samsung 990 Pro, rated 600 TBW, dead at 60.
Another: SSD failed at 3% of expected lifespan. Another: 0.5 TB written every single day.
If you run Codex 24/7:
- Update now
- Check SMART (smartctl or CrystalDiskInfo)
- Any wear already logged is permanent
Hardware consequences from software bugs won't be rare anymore.
⚠️ Waller put a lot of emphasis on inflation breadth. June CPI was undoubtedly soft from a breadth perspective - only 43% of the basket running above 2.5% annualised inflation (lowest since Feb '25). 47% of the basket saw -ve %MoM prices... the most since Aug 2024. Doves win $USD
I have made the following paid articles free.
I believe basic knowledge of optics, especially CPO, is something everyone should understand, so I decided to make all of my previously paid articles on this topic available for free.
I can confidently say that if you read all of these articles, you will be able to build your own conviction on CPO.
Please stop by my Substack, subscribe, and enjoy the articles below.
The War of Light Has Begun: https://t.co/AApOwRrrE2
The War of Light, Part2 - Who's at the Table, Who's Not, and Who's Hiding: https://t.co/NYIiSW9EDg
Why CPO Is Becoming inevitable? [CPO Special Part1]: https://t.co/dTuQLNYrjL
CPO, Fully Dissected [CPO Special Part2]: https://t.co/4pe27hwVbh
The Illusion of CPO [CPO Special Final]: https://t.co/D3dTDXOJ58