Erik Bernhardsson

Building everyone's favorite AI infrastructure platform @modal

[Erik Bernhardsson @bernhardsson]

Deep and fun conversation on @latentspacepod about the cloud market, why sandbox demand is exploding, why code reviews suck, and much more. Thanks for having me on!

[Basil Chatha @realbasilchatha]

Last week, @bernhardsson (Founder & CEO, @modal) announced a $750M Series D. I sat down with him the prior week to discuss a ton of stuff about the business.

We got into:
-why open-source models made fine-tuning a lot less necessary
-how Modal's team grew 4x in a year and could hit ~700 people next year
-why the frontier labs have the same business model as music artists
and much, much, more…

Enjoy!

Timestamps:
0:00 What Modal does
1:25 From @Spotify to building @modal
4:19 Founding before the generative AI boom
7:43 First customers and fundraising
10:00 Building for inference and autoscaling
13:21 Where Modal gets GPU capacity
15:39 Renting vs. owning infrastructure
19:45 The risk of GPU prices falling
24:18 Is AI demand infinite?
28:25 What Modal can learn from neoclouds
30:26 GPUs as an asset class
31:55 Modal’s products: functions and sandboxes
36:03 The case for smaller training workloads
41:21 How agent sandboxes are changing
46:48 @AnthropicAI and sandbox security
49:29 Using agents to test security
52:51 Developer experience in the age of agents
57:45 What Modal’s forward-deployed engineers do
59:55 Modal’s approach to product management
1:03:29 Optional code reviews and engineering culture
1:05:56 Cloud, on-prem and bring-your-own-cloud
1:09:25 Capacity, fundraising and growth
1:13:07 Scaling the team
1:17:28 Erik’s tweets: ephemeral sandboxes
1:18:31 The problem with rectangular GPU contracts
1:21:20 Inference providers vs. model builders
1:23:25 Where models improve next

[Erik Bernhardsson @bernhardsson]

@saimonsharif @latentspacepod I have many interests

[Erik Bernhardsson @bernhardsson]

Think we’ll come out of the winter with really good defensive use of AI for security though.

[Erik Bernhardsson @bernhardsson]

It’s going to be a long winter for every security team out there. Be nice to them.

[Daniel Shaar @dshaar_]

Some projects are only done for shareholder value. This one was done for the love of the game, literally! My favorite online game, Curvytron, is now backed by the Sticky Sessions primitive my team recently shipped :)

https://t.co/bmgf9A680t

Shoutout to @siwonpark_, @ksinpoint1, and @kathirmeyyappan for embodying the @modal spirit and shipping this with speed.

[Erik Bernhardsson @bernhardsson]

@JDAudena @modal Last month revenue times 12

[Erik Bernhardsson @bernhardsson]

@matt_slotnick @JustJake pretty hard to move up the stack imo

[Erik Bernhardsson @bernhardsson]

It can be simultaneously true that we’re going to need a shit ton of GPUs and that the equity return of owning GPUs isn’t going to be super high. I’m biased but I think most value will accrue in the software layer – applications and infrastructure.

[Erik Bernhardsson @bernhardsson]

@shomikghosh21 A bit hard to say what the equity returns were given that in all cases they were part of a bigger company.

No doubt the revenue growth has been incredibly impressive though.

[Erik Bernhardsson @bernhardsson]

@moinnadeem I think neoclouds may be an ok investment tbh. Just think share prices won’t reflect revenue growth

[Erik Bernhardsson @bernhardsson]

@moinnadeem a) neither – share price. Note that neoclouds are issuing quite a bit of equity, diluting shareholders
b) think no (esp consider how hotels or airlines often lease don’t own assets)

[Erik Bernhardsson @bernhardsson]

@_dylanga 💚

[Erik Bernhardsson @bernhardsson]

@JustJake I think that intuition is right, though I think we’ll be supply constrained for quite some time.

[Erik Bernhardsson @bernhardsson]

@alexgshaw I think there’s going to be massive revenue for sure, but not sure about equity returns. Training is expensive.

[Amit @amitpr]

The secret to Modal's speed is our custom lazily loaded container filesystem, ImageFS. ImageFS is built on a massive tiered distributed filesystem, layering local NVMe disks, regional and p2p caches, and global blob stores.

When we launched VM Sandboxes this month, we wanted to make sure they could take advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.

[Erik Bernhardsson @bernhardsson]

@venkat_systems @antonosika I’m actually obsessed with ironing

[Erik Bernhardsson @bernhardsson]

@nikhilvnamburi They will like it until I tell them my feelings about predictability and forecasting!

[Erik Bernhardsson @bernhardsson]

@nikhilvnamburi Putting it in opex is borderline accounting fraud imho