dan

building edge compute for inference

[dan @danjjensen]

“Even if the entire population of Earth donned motion-capture suits, it would take decades to generate the amount of data that was used to train ChatGPT”

There’s a lot of hype about all the robots coming over the next several years (1 billion according to Musk), but seems like the data gap has a long way to go until it’s closed.

https://t.co/Wi2KPPQS9i

[brexton @brexton]

The scarcest and most in-demand resource on earth right now is GPU financing for sub-IG/unrated offtakers and ASICs

This tweet will appeal to approximately 6 people

[dan @danjjensen]

VCs don't want to be a neocloud. Leave the cluster ops to a competent operator-provider.

[Sonnet Xu @sonnet_xu]

VCs (should) want their own compute. Do they actually want to be a neocloud?

I've chatted with a few different VCs owning/operating their own clusters, but they don't have the infra team or the operations team to deal with issues from cluster uptime to pricing. Some funds, especially those with neocloud connections, effectively outsource the whole thing to a neocloud. @ycombinator's dedicated cluster, for example, is operated by @togethercompute. Others are closer to owning the underlying capacity themselves, finance + built it, and need an operations layer on top.

Several startups are already looking into operations as a service for VC compute. Separating ownership from operation allows for capital exposure to compute + for higher liquidity and margins (VC clusters often don't have long term locked in contracts). Splitting financialization from logistics is just the start.

[Sonnet Xu @sonnet_xu]

VCs (should) want their own compute. Do they actually want to be a neocloud?

I've chatted with a few different VCs owning/operating their own clusters, but they don't have the infra team or the operations team to deal with issues from cluster uptime to pricing. Some funds, especially those with neocloud connections, effectively outsource the whole thing to a neocloud. @ycombinator's dedicated cluster, for example, is operated by @togethercompute. Others are closer to owning the underlying capacity themselves, finance + built it, and need an operations layer on top.

Several startups are already looking into operations as a service for VC compute. Separating ownership from operation allows for capital exposure to compute + for higher liquidity and margins (VC clusters often don't have long term locked in contracts). Splitting financialization from logistics is just the start.

[Liquid Compute @liquidcompute]

We spent the last few months talking to credit investors about financing GPU fleets. Four questions came up in nearly every conversation:

1. What does the market actually price?
2. Is there a secondary market for the hardware?
3. How do you tell good offtake from bad?
4. What does the depreciation curve look like?

No document answered them, so we wrote one.

Investing in the Middle Market for Compute covers the segment below the hyperscale tier, where the borrower is a real operating business and the offtaker is not investment grade. The core finding is that capital is not the scarce input here; bankable offtake is. Operators routinely hold term sheets they cannot draw because the layer beneath the debt is unfilled.

It also covers the down payment gap that kills deals, how the market views obsolescence, and why recovery in this asset is a distribution problem. Plus observable pricing from our own book and the security package items operators will actually give you.

If you'd like a copy, shoot us a message and we'll send it over.

[Ivan Burazin @ivanburazin]

CPU clouds are the next wave. In about 24 months, you'll see it around you.

Everyone optimized for GPUs because training and inference are gpu workloads.

But think about what agents typically do:
- running code
- clicking buttons
- reading files
- calling APIs
- writing outputs

All of these are CPU workloads. The ratio of doing to thinking for a productive agent is probably 10:1 or higher.

The compute spend on agent execution will eventually exceed the spend on inference.

[danning @sui414]

GPU prices differ fundamentally because you’re buying under different financial contracts.

(besides other reasons - specs, market fragmentation, and quoted vs. cleared prices.)

From all the public prices we audited, there are roughly 4 pricing tiers in the market today:
1⃣Spot = interruptible excess capacity
2⃣Reserved = 3mo / 6mo / 1yr+ commitment
3⃣ “On-demand” = pay-as-u-go, non-interruptible
4⃣Managed inference = GPU + serving stack

You can clearly see the term discount relative to on-demand, as the buyer commits upfront and takes on more price/utilization risk.

And managed inference commands another premium for the serving and optimization layer on top.

This is the same chip, but 4 different products' pricing range:

H/t @waylonjepsen for the guidance & discussion!

[dan @danjjensen]

@RichLowry Tell us more, for those less familiar with the history

[dan @danjjensen]

@WeatherDoogs Like this one!

[dan @danjjensen]

@gromerjeffers Midwestern manners = make sure everyone feels welcomed and appreciated..even the media!

[dan @danjjensen]

@DavidAFrench Here’s my question: the current iteration of our national flag was established in 1960. But this was during the era of Jim Crow and segregation. So, is it too a symbol of white nationalism and bigotry?

[dan @danjjensen]

@nytopinion Greatness is not measured in sets of data, but in stories of sacrifice and a commitment to strive for high ideals. No people have sacrificed more, or been more committed to the cause of freedom and justice than the American people. This is the greatness of America.

[dan @danjjensen]

@nytopinion The greatest of American lies in its ideals, chief among these that men are are permitted to rule themselves. Whatever our destiny (or rankings) is because we willed them to be. Ours is a country-and a government-of and for the people. We decide. And that is the whole point.

[dan @danjjensen]

@CityJournal @SethBarronNYC The same is becoming true of Boston Common and the Public Garden. You cannot scroll through the park without inevitably walking through plumes of smoke or being witness to a transaction.

[dan @danjjensen]

@TexasTribune Depends who Ds put on the top of the ticket. Had @BetoORourke chosen to run for Senate again, could have helped in a big way to that effort

[dan @danjjensen]

Social capital and economic opportunity go hand-in-hand. Entrepreneurship flourishes in areas where social capital is highest. Vibrant communities serve as launching grounds for risk taking and, ultimately, more prosperity.

[City Journal @CityJournal]

A shadow lingers over the country’s otherwise bright economic outlook. It’s not a shadow cast by the trade war, volatile stocks, or inequality, but rather by low social capital—the interpersonal relationships that generate a shared sense of community. https://t.co/hO3O35DwQE

[dan @danjjensen]

@PatrickSvitek New comer to Texas — moved here in ‘16. Guess that makes him squarely an Austonian?

[dan @danjjensen]

Lets talk about the federal budget and this astonishing fact: 80 cents of every dollar of individual taxes (Income + Payroll taxes) goes to pay for three federal programs: Social Security, Medicare, and Medicaid.

[dan @danjjensen]

@RichLowry And here is the promise of socialism: Relinquish your liberty to the state and I will make you free.