Substrate

Researched writing and conversations on frontier technology.

[CAT @cattttang]

the conventional view is that datacenters require reliability

but if your workload tolerated 95% uptime? 90%? you could move workloads between sites if the dc drops and take advantage of underutilized (cheaper) capacity

[Substrate @substratemag]

So long as the frontier labs charge premiums on tokens, they can always outbid startups for compute - for now, NVIDIA's goodwill keeps the ecosystem running

[CAT @cattttang]

I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.

"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."

Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.

[Substrate @substratemag]

On modular datacenters:

NVIDIA has been piloting modular (5-20mw) datacenters, and SemiAnalysis predicts they'll make up 31% of capacity by 2028.

A big reason why is lead time. 50mw-100mw datacenter lead times are 12-24 months. For <1mw datacenters, 16 weeks

[CAT @cattttang]

I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.

"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."

Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.

[Substrate @substratemag]

"There's no stranded compute for us anymore."

Most inference providers avoid stranded capacity. Its reputation for unreliability is well-earned. But if you drop latency requirements for background workloads, you'll unlock pools of compute no one else has at prices no one can beat.

Interactivity is expensive. Throughput is cheap.

@cattttang spoke with Sail Research co-founder Neil Movva on latency-insensitive inference, alternative chips, and <1MW datacenters.

Full conversation below.

[CAT @cattttang]

I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.

"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."

Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.

[P Equity Research 📰 @pequityresearch]

I find it surprising that less than 5% of SemiAnalysis' revenue comes from Substack. They are incredibly popular...thought it would be higher.

[Substrate @substratemag]

@SemiAnalysis_ has be AI-pilled to have the right priors

A few years ago, no one thought demand would be able to hit today's numbers

In 2023, while modeling datacenter builds, one team member said, "There was a thought of 'Did we mess anything up on the numbers?'....I asked the team, 'How confident are you in these numbers?' They were pretty damn confident, so I just worked on that assumption."

[Substrate @substratemag]

Full post here: https://t.co/fYFQplpTWn

[Substrate @substratemag]

@SemiAnalysis_ rose to fame from its Substack, but the newsletter only makes up &lt;5% of revenue - their product expansion into models, institutional research, benchmarks, etc. has been massive

[Substrate @substratemag]

@SemiAnalysis_'s early bet on predicting NVIDIA's revenue via TSMC CoWoS output and earnings calls from tool companies kickstarted their model program

The accelerator model is one of the most popular today

[Substrate @substratemag]

In just a few years, @SemiAnalysis_ has risen to the forefront of AI infrastructure information, with their work referenced by some of of the biggest players in semiconductors.

The prediction: "@dylan522p's going to be the greatest analyst of this generation."

[CAT @cattttang]

I sat down with @dylan522p and spoke with over a dozen people about his information machine.

"I don't think anyone who has a traditional equity research background would even try to figure out where $3T in [the AI labs' combined] ARR comes from."

Slack private messaging limits, finding CoWoS volume, and the industry they're looking at next - I wrote about SemiAnalysis here: https://t.co/0FVDIJNaGe

[Substrate @substratemag]

Welcome to Substrate Magazine. Substrate covers frontier technology, from RL algorithms to DRAM supply chain.

@SemiAnalysis_ is the leading information firm in the largest infrastructure buildout in history.

We're excited to profile @dylan522p for our first piece.

[CAT @cattttang]

I sat down with @dylan522p and spoke with over a dozen people about his information machine.

"I don't think anyone who has a traditional equity research background would even try to figure out where $3T in [the AI labs' combined] ARR comes from."

Slack private messaging limits, finding CoWoS volume, and the industry they're looking at next - I wrote about SemiAnalysis here: https://t.co/0FVDIJNaGe