5/5 Data Center Expert Call | Hyperscaler ASIC Economics vs. Merchant GPU
Custom silicon (Google TPU v6/v7, Meta MTIA, AWS Trainium3) is not replacing merchant GPUs across general workloads; it is serving predictable high-volume inference pipelines:
- Packaging cost breakdown: A 3.5x reticle CoWoS-L substrate with 8 HBM3e stacks carries a raw silicon and packaging BOM of $4,800 to $5,300 before test and yields.
- Breakeven threshold: Custom ASIC internal programs require lifetime volume exceeding 450,000 to 500,000 production units over a 3-year amortization window to offset $300M+ in 3nm/2nm tape-out, software stack maintenance, and non-recurring engineering (NRE).
- Allocation reality: Hyperscalers dedicate custom ASICs to internally owned, fixed-graph workloads (recommendation rankers, target embedding generation, proprietary foundation model decode). Elastic, multi-tenant public cloud instances remain 75%+ committed to merchant GPU clusters.
1/5 Data Center Expert Call | Grid Interconnection & Power
The hard ceiling on tier-1 US data center buildouts is not capital; it is substation step-down capacity.
- Large Power Transformers (LPTs, 345kV/500kV): Lead times sit at 145 to 160 weeks. Tier-1 utilities (PJM, ERCOT) have pushed firm substation interconnect studies to late 2028.
- Behind-the-meter bridge power: Hyperscalers are deploying aero-derivative gas turbines (GE Vernova LM6000, 50MW blocks) at $1,400 to $1,600/kW installed cost to bridge 3-year utility delays. Fuel supply contracts require dual-fuel capability (natural gas plus 48-hour on-site diesel storage).
- Power density migration: Legacy 15kW/rack rooms cannot be retrofitted economically. New campuses standardize on 120kW to 140kW per rack footprint, pushing campus requirements from 100MW to 600MW dedicated capacity.
4/5 Data Center Expert Call | Memory Hierarchy & KV Cache TCO
The 800:1 to 1,000:1 FLOPs-to-bandwidth imbalance between prefill and decode stages is forcing compute architectures apart:
- Prefill nodes: Compute-bound, optimized for peak BF16 tensor core utilization with standard HBM capacity.
- Decode nodes: Memory bandwidth and capacity-bound. Storing 128k context KV cache states in HBM costs roughly $22 per gigabyte.
- Tiered retention: Production clusters are validating a three-tier hierarchy:
1. Active tokens in HBM3e (8 to 9.6 TB/s bandwidth).
2. Idle KV blocks paged out via PCIe Gen 6/CXL 3.1 to host DDR5/LPDDR5 memory ($4/GB).
3. Persistent session state written to 64TB/128TB QLC enterprise SSDs ($0.08/GB) over NVMe-oF, testing dual-actuator HDD arrays for cold archival retrieval under relaxed 200ms TTFT constraints.
Spot on. When prefill dwarfs decode by 800:1, inference stops being a compute problem and becomes an inter-rack memory streaming problem.
This is forcing the architectural push toward Prefill-Decode disaggregation: dedicate high-bandwidth pods to hold the prefix state, then stream compact activations to cheap decode workers.
It's also why Google went all-in on optical circuit switching (OCS) for TPU clusters: you either route the light dynamically, or the interconnect tax eats your ASIC silicon margin.
Prefix caching and agentic loops push prefill-to-decode ratios past 800:1, shifting TCO to KV residency. Monograph #4 breaks down the hardware and thermal forces driving the shift to modular inference ASICs. https://t.co/9m2jbrP9Sm
Thanks Jack. Component-level pricing debates always miss the forest for the trees.
At 200G/lane PAM4, the margin for error is measured in femtoseconds and milliwatts. Hyperscalers pay for proven signal integrity and production runtime, not pitch deck claims. Looking forward to more deep dives on the physical layer with you both.
Why Intel insists High-NA EUV is 'cost-effective' while TSMC demands 3 major tool upgrades before touching it:
Intel's 2022 pre-order of ASML EXE:5200 was Gelsinger's defensive reaction to Krzanich missing Low-NA EUV on 10nm. It was a corporate identity hedge, not a fab cost optimization.
Today, engineers are forced to justify half-field stitching and low initial WPH by claiming Backside Power Delivery makes the math work.
TSMC isn't carrying that psychological baggage. Without PR pressure, Morris Chang and CC Wei simply held the line with 0.33 NA multipatterning until High-NA source power and overlay mature.
Can the Intel gurus help me translate this? Are we saying here that high-NA EUV (EXE:5200) is cost-effective for 18A w/ BPS?
Because at 16/14A everyone has BSP. And we have TSMC over here saying at least 3 major tool upgrades are needed before they adopt it, 1 stated upgrade and 2 implied based on the timing.
TSMC's 3 tool upgrades aren't arbitrary: they need >500W source power, >200 WPH, and sub-1.1nm machine-to-machine overlay before the Capex per good die beats 0.33 NA.
Intel is eating early depreciation on EXE:5200 for PR yield milestones, but running half-field stitching in volume manufacturing will hammer gross margins until tool maturity catches up.
Hyperscalers won't risk a $500k training checkpoint rollback to save 5% on intra-rack cable BOM.
AEC qualification cycles run 18+ months for a reason: field MTBF across millions of deployed links is the only metric that matters. $MXL entering 1.6T expands the total addressable market narrative, but $CRDO's production lock-in remains unthreatened.
The bigger buy-side takeaway here is the timeline: pushing HBM hybrid bonding to 2029+ completely vindicates SK Hynix sticking with Advanced MR-MUF through 16-Hi HBM4.
Micro-bump physics still have runway, and Hanmi TC Bonders aren't getting displaced anytime soon. $ASML $000660.KS
Holding uncontracted, fully energized megawatts is the ultimate call option in a 4-year transformer lead-time environment.
$IREN can demand front-loaded capex contributions because buyers can't wait on utility queues. The only caveat to monitor is SLA execution penalties if modular substation commissioning hits delays.
@jukan05 Jensen running the ultimate Active Thermal Control test before the Rubin tape-out. If this is what 'peak cycle' looks like, book another 50k wafers at TSMC.
Chroma ($2360.TW) SLT revenue ramping from NT$4B (2025) to NT$9B (2027E) marks the structural shift from single-GPU dependency to platform diversification:
Multi-program ramp: Google/Marvell inference ASIC + Vera CPU + Rubin Ultra in 2H27
Physics driver: 1500W+ module TDP mandates high-precision Active Thermal Control (ATC) liquid handlers
Extended test duration per package directly multiplies tool unit requirements.
$2360.TW: Bernstein maps a 2027 CPU and GPU testing ramp
Bernstein expects Chroma ($2360.TW) to see 2027 system-level testing demand from $GOOGL/ $MRVL inference projects and $NVDA’s Vera CPU. It also forecasts Rubin Ultra GPU testing to ramp in 2H27.
That gives $2360.TW several potential program ramps to track, rather than a single AI-chip catalyst
The durability argument holds up because hyperscaler HBM 3:1 wafer cannibalization leaves zero margin for the Big 3 to re-enter DDR3/trailing DDR4.
At 1.6x runrate P/E, the key risk isn't pricing, it's PSMC wafer allocation ceilings. If their 19nm tape-out ramps smoothly into 2027, gross margin expansion survives even if foundry input costs climb further. $ESMT
6/6 [KV Cache & Physical AI] Beyond training logs, WDC is sampling dual-actuator high-bandwidth HDDs to hyperscalers to evaluate substituting expensive DRAM/SSD tiers for inference KV cache workloads.
Top 7 customers provide demand visibility to 2030, but near-term margin debates and CapEx creep will test valuation multiples in a risk-off tech tape. $WDC
1/6 Western Digital ($WDC) reported Q1 FY27: Rev +55% YoY, EPS +161%, gross margin 58.1%, and $1.4B FCF. Yet shares fell -12% across regular and after-hours trading.
Behind the sell-the-news tape: an Exabyte growth gap vs peers, head wafer bottlenecks, and CapEx intensity inflections. Key Q&A takeaways below. 🧵👇
5/6 [Head Wafer Physics & CapEx] Management reiterated zero incremental unit capacity additions: bit growth is 100% areal density driven.
However, HAMR magnetic heads require significantly more process steps and longer fab cycle times, creating throughput loss. To support HAMR heads and future 14-platter drives, CapEx will track above the historical 4-6% revenue baseline for the next two years. $WDC