Chad Wallace

Deep-Tech Semiconductor Analyst, MSEE @JohnsHopkins. Expert-driven system architecture deep-dives of AI datacenter hardware at

[Chad Wallace @siliconcodesign]

Most people who use AI and invest in AI Infrastructure operate at a high abstraction level and chase trends.

However, developing a deep and broad understanding of AI Infrastructure is difficult because domain-specific technical data lives in isolated silos and isn't translated well to a broad audience.

As a result, engineers often don't have system context of what they are designing and investors often don't have a decent mental model of what they are investing in.

My goal is for investors and engineers to think more like system architects, because architecture decisions have financial implications and influence day-to-day execution of engineering teams.

Over the past year, I've travelled to the semiconductor industry's flagship conferences and learned how these subsystems worked from the industry's foremost experts.

I've written a complete suite of mental models on the subsystems that comprise AI Infrastructure:
- Mixed-Signal ICs (SerDes, ADCs, PLLs, Bandgaps)
- High Speed Optical/CPO
- High-Power PMICs
- High-Speed Signal Integrity
- Advanced Packaging
- AI for Chip Design / EDA
- AI Accelerators
- Memory (HBM and SRAM)

Link to the website is in my bio and in the comment below 👇

[Chad Wallace @siliconcodesign]

Jensen of @nvidia , Lisa Su of @AMD , and Lip-Bu Tan of @Intel_Foundry at @TomorrowXSummit Nov 17-18 in Austin, TX

[Chad Wallace @siliconcodesign]

I think people focus too much on the CapEx Investments in AI infra and forget about that skilled labor that CapEx helps pay for.

At the end of the day, the capital is being used to pay people, and leaders/investors need to invest in training and instill a sense of purpose for those people.

[Chad Wallace @siliconcodesign]

@Silicon_Atlas Honestly, I think the labor needed to build out the infrastructure is the big bottlenecks, since at the end of the day, the capital is being used to pay people.

[Chad Wallace @siliconcodesign]

AI Compute Racks need three things to process tokens and survive: Power, cooling, and data.

To maximize compute density, the plumbing for all three domains need to be co-designed to work safety and reliability with each other.

Next week, I will be releasing my masterclass on co-designing power, cooling, and networking architectures from @OpenComputePrj summit.

This post will deep dive into important tradeoffs in each domain. It will also show how the throughput of all three domains need to scale alongside with compute power to maximize performance and minimize bottlenecks.

[Chad Wallace @siliconcodesign]

Really great to see Dwarkesh getting into Hardware!

[Dwarkesh Patel @dwarkesh_sp]

I spent two days touring Intel’s most advanced fab and talking to its technical leadership to understand what the chips of the future will look like.

A lot of what's coming next – gate-all-around transistors, backside power, CFET, hybrid bonding – is hard to follow without actually seeing it, so we made all the awesome animated explainers that I wish I had while I was prepping.

I also asked Intel’s leadership how the semiconductor industry will cope with the enormous shock in demand from AI.

AI compute has grown 3x a year for the last decade, but according to Intel's CTO, Intel could only double leading-edge wafer production with today’s global stock of chipmaking tools.

What kinds of tricks can foundries pull to extend this crossover point where demand surges way ahead of supply?

This was a totally new format for us - hope you enjoy!

0:00:00 – Intro
0:00:47 – How Intel brings up a new process node
0:03:22 – What Intel learned from 10nm
0:06:28 – From planar transistors to gate-all-around
0:13:13 – Why High-NA EUV pays off for Intel but not TSMC
0:19:20 – Could chips have evolved completely differently?
0:23:33 – Inside Intel's "Hot Topics" meeting
0:29:30 – Replacing copper wires with ruthenium
0:32:41 – Why stacked transistors take a decade to ship
0:39:45 – Hybrid bonding and the chip-to-chip bottleneck
0:48:00 – There aren't enough tools to 10x chip production
0:53:14 – Can Intel catch up to TSMC?
1:01:02 – Terafab, and rethinking the fab from scratch

[Chad Wallace @siliconcodesign]

The Halo effect:

[Molly O’Shea @MollySOShea]

Ken Griffin of Citadel questions the economics of the AI infrastructure boom:

›What happens when GPU supply exceeds demand & pricing goes into free fall?

›Are today's semiconductor margins sustainable? Are investors overestimating the useful life of GPUs?

“Look, I think this market is actually experiencing people doing research.

For the couple of years, if you were on the thematic program of we're gonna spend hundreds of billions or a trillion dollars in data centers, we're gonna build compute at a scale never seen before, which companies in that story are choke points in supply chains that will have abnormally high margins? And who will be the big winners in this transformation?

Will it be the Microsofts & AWSs who will be the gatekeepers to the foundational models? Like, who's going to earn the economic rents?

But in some sense, like, 'I'm just gonna buy them all. I'm gonna buy the hyperscalers, I'm gonna buy the chip, I'm gonna buy businesses that make things I've never even heard of 5 years ago in Korea and Taiwan & in Japan.'

And now we're reaching a point where people are becoming more discerning.

We do realize that we are seeing extremely high margins in DRAM & MLCCs in areas where there are real supply constraints, but people are now starting to look out 2 years, 3 years. How long is this going to last?

And in particular, I don't wanna be doom & gloom, but GPUs, like airlines, have a very low marginal cost to run.

The incremental seat on an airline that's not full, if you can sell that ticket for $20, you're making money in that seat.

And that's one of the reasons that the airline industry is such a nightmare, is because the moment you have overcapacity, pricing goes into free fall.

We saw this actually with the telecom build-out back in 2000. We built more fiber than we had demand for, & the price of telecommunications just went straight down.

If we reach a point where the provisioning of GPUs exceeds demand for GPUs, marginal pricing will actually go into free fall.

And that really upends all the investment assumptions that people are working with today.

That's issue number one.

And issue number 2 is:

I think people are taking a step back & saying, 'What is the useful life of a GPU?'

Right here, right now, there's people that are earning extraordinary pricing on GPUs that are 5 or 6 years old.

And so there's a growing set of concerns that I can't give the same capitalization rate or the same revenue models to year 7, 8, 9 of a chip in the future that maybe I was using 12 months ago.

So I think we're really grappling with just how far will this demand carry us in the marketplace.”

[Chad Wallace @siliconcodesign]

@sallywf Been there myself :) Make sure you check out Trattoria Zà Zà

[Chad Wallace @siliconcodesign]

@sallywf Ponte Vecchio bridge 😃

[Chad Wallace @siliconcodesign]

@iamfabian You're like "The Diplomat" in real life 😄

[Chad Wallace @siliconcodesign]

The six materials that shape and power AI datacenters - sand, salt(?), iron, copper, oil, and lithium

[Chad Wallace @siliconcodesign]

Agreed

[Vance Spencer @pythianism]

Chads are in

[Chad Wallace @siliconcodesign]

@bookwormengr Yeah that's like the whole theme of ISSCC 2027

[Chad Wallace @siliconcodesign]

How JEDEC standards are an impediment to pushing memory BW and the technical fundamentals needed to co-design HBM for $MU :

https://t.co/2EgXwINiSB

[Chad Wallace @siliconcodesign]

The key processing steps of hybrid bonding, how thermo-mechanical stress makes it hard to hybrid bond multiple layers of HBM die, and key suppliers (besides BESI):

https://t.co/1MUhESdz6E

[Chad Wallace @siliconcodesign]

@leopoldriskmgmt lol it’s one thing to say you can do hybrid bonding, it’s another thing to deal with thermomechanical stress over high volume