Olivia
Online
Sonnet Xu
@sonnet_xu
cs @stanford
Joined January 2025
18 Following
67 Followers
26 Posts
virtualization choice is one of the biggest hidden variables in the GPU rental market
a few common ways are (1) bare metal/passthrough, (2) container on a shared host (think Docker containers, what marketplaces like https://t.co/SXm1IAkK47 do (3) time-sliced vGPUs, where you can run into the noisy neighbor problem
it's interesting because the provider market has split with very different architectural philosophies
Nebius is hypervisor-first and quite transparent
CoreWeave has no hypervisor and touts bare-metal Kubernetes
if you're in deep tech/hardware/robotics (or know someone who is) and might want to try out venture, a really great + technical early fund (you've heard of them before) might want to chat ! looking for engineers/researchers 🙃
reach out if you want details :)
the knowledge vs. safety priority for benchmarks in medicine is such a great questions and cool to see new models thinking about this!
Generic AI models are not built for medicine.
Doximity Ask, trained and served on Fireworks, outperformed GPT-5.6 Sol, Claude Fable 5, and OpenEvidence in a Stanford-Harvard clinical AI safety study.
Specialized intelligence beats the generic version when safety matters. @doximity proved it.
It's great to see more work done in GPU reliability, super excited to see how Slurm on Kubernetes evolves. Every one of Together's five fixes maps to a specific mismatch between Slurm's working assumptions (owning the machine, running forever) and Kubernetes' assumptions (ephemeral workloads, ownership of process tree, cgroups, and devices)
while leaders are pushing out updates, there's a long tail of smaller neoclouds who have nothing built out yet...
We made Together GPU Clusters more reliable and easier to operate.
Passive health checks, guided node repair, a rebuilt Slurm stack, better cluster visibility, external OIDC, startup scripts, and acceptance-test controls are now live.
Learn what shipped 👇
https://t.co/M6khfMJ0w2
Crusoe frames itself as the industry’s first vertically integrated, purpose-built AI infrastructure provider.
As ex-crypto players who used to work on flare-gas, there's a deep understanding of the energy and full-stack, from land to cloud
recently sold off their crypto arm while it was profitable, excited to see what's next
Gartner® named Crusoe a Visionary in the inaugural 2026 Magic Quadrant™ for Cloud AI Infrastructure.
Most AI teams hit the same wall: too little compute, too much infrastructure complexity. We built Crusoe to remove both.
More from our SVP of Product Management: https://t.co/IsLuwnh5W2
@Gartner_inc #AIInfrastructure #AICloud
love the point on nonfungibility, since there's so much that goes into GPU performance and pricing, from hardware to interconnect to how the provider supports and virtualizes and more!
People keep comparing compute futures to oil, but the correct comp for market structure is actually electricity futures
Expect wide basis risk with supply step functions rather than cash and carry storage arb and the central battleground will be nonfungible index methodologies
i only feel safe using fable 1 hr before my session is going to reset :(
The thing that's frustrating is Fable quickly saturates your current session limits, blocking you from using any other models in your subscription!
It makes the subscription significantly less valuable because you can get easily caught by 1 of N limits
We see a version of this in GPU compute, where buyers want to hedge provider reliability
Once you can measure the risk well, a market or parametric product on top of it feels almost inevitable
Really really cool writeup from the one and only @lzminsky. Pred markets as a mechanism to manage corporate risk is an enormous opportunity + development.
The core question is creating the right product structure - one that takes in SMB data to identify actual risk vectors, relevant markets + quantify basis risk.
A really interesting path to create DeFi-native "insurance" products, and always feel inspired by my convos with Lauris!
would be super interesting to see how the faster token gen benchmarks holds when you take the heterogeneity of compute into account
compute isn't uniform and it varies a lot!
Thanks for the support @AndrewYNg! Completely agree, faster token generation will become increasingly important as a greater proportion of output tokens are consumed by models, such as in multi-step agentic workflows, rather than being read by people.
"compute is still not uniform enough"
excited for the grading and benchmarking layer to come
Today, we launched GPU compute forward curves derived from our prediction market prices. Forward curves are now available on Nvidia B200. H200, and A100 chips.
Forward curves track implied future prices. They are how mature commodity markets form expectations, allocate capital, and manage risk. Energy, interest rates/SOFR, FX, metals, and agricultural markets all rely on market-implied forward prices.
Despite becoming one of the key inputs in the global economy, compute has lacked that market-derived infrastructure. Compute right now is where oil was before NYMEX — traded only via OTC deals, just like oil used to trade OTC between producers and refiners. As compute becomes as fundamental to the economy as energy, the industry will need a similar derivative market to promote efficient price discovery.
Prediction markets are uniquely suited to this problem. Compute is not one uniform commodity and spans many chips, grades, tenors, locations, and contract structures. A live prediction market can aggregate those dispersed views into transparent prices that reflect market expectations for different maturities.
The opportunity is big. Hyperscalers are spending over $700B on compute this year and the market is expected to grow to $7-10T by 2030. If this market behaves like traditional commodity markets, a liquid derivative market could be 10-20x bigger than the underlying spot market.
Compute is still not uniform enough, but this is a step towards standardization as forward curves will help us see the rise and fall of different model prices and how they correlate.
The forward curve is a first step. Up next: futures and perps.
Lots of compute providers moving to offer managed fine tuning etc
Clusters break all the time, neoclouds don’t quite have the incentives or SLAs to keep them running
Fine-tuning used to mean spinning up a cluster and hoping nothing broke before the job finished. Then you got to do it again for deployment.
Crusoe Serverless Fine-Tuning and Self-Serve Deployments are now GA. Tune an open model, deploy it to production, no GPU cluster to manage.
Read more → https://t.co/O6rIJb2VbK
#FineTuning #OpenModels
an interesting piece to add here that isn't the same is how the compute health and performance evolves over time! another critical thing to think about
Today we're launching https://t.co/BlKNFocVzu.
Buying or selling GPU capacity means describing the same cluster over and over: model, count, region, fabric, term, price. It's slow to write, easy to garble, and impossible to track once it's been forwarded a few times.
this seems super interesting! def a lot to learn here
CoreWeave's SLA is a credit-only floor that doesn't cover what actually breaks training. Its 99.9% tier only pays if you're redundant across 2+ regions AND all go down at once (not really training realistic...). So they now sell straggler detection as a feature instead of insuring uptime.
at least they have an SLA though, most neoclouds don't :(
At scale, failures aren't rare - they're routine. A node drops, a GPU faults, and your recovery costs quietly compound. 📉
Join us on July 21st for Episode 2 of Training Tuesdays to break down the real cost of downtime and how to optimize your recovery workflows.
Secure your spot here: https://t.co/GcOvtnmEI7
agreed that the rate isn't the cost
the thing with serverless though is the node you draw is a big part of it too. it'd be interesting to think about how variation is across nodes
we ran our a small test on modal's serverless 8x h100 three times. running the same small training step on each of the eight gpus, the slowest card took about 4x longer than the fastest on every run, roughly 83ms per step against 22ms
training throughput also moved with the node, from about 19k to 22k tokens per second between two otherwise near-identical draws, and lower on the third.
would be fun to fold a node-quality factor into that widget!
Rates are not costs!
Serverless GPUs can cost more per hour but in many practical cases they cost less in aggregate.
The key statistic is the workload's peak-to-average demand ratio.
I wrote a lil article for the @modal blog (and vibed up a lil widget) demonstrating this.
I started at Stanford spending a lot of time thinking about and researching how to benchmark LLMs, now I’m really excited about seeing how we can benchmark GPUs! Benchmarking compute is next!!
“The dirty secret in AI is that everything is a data and an eval problem.
The best models have the best data and best internal benchmarks. The mid ones buy a lot of data, not the best, and hillclimb public benchmarks.
(you need a lot of compute too)”
– Stanford CS Professor
For decentralized GPU marketplaces, benchmarking and understanding heterogeneity within the cluster is even more important
64 H100s for 10 days is not a benchmark slide. It is a cash-flow event. In one quote, decentralized GPU supply priced 40,032 USD below AWS for the same workload class. That delta is another training run, longer eval, or one more engineer-month. Takeaway: GPU cost is product velocity.
Centralized cloud optimizes for enterprise allocation. Capacity sits behind quota tickets, committed spend, regional scarcity, and sales priority. For small AI teams, budget, code, and demand can exist while launch still waits on approval. Takeaway: the bottleneck is not only silicon; it is access control.
Decentralized GPU flips inventory. Idle global GPUs become elastic supply. Dynamic pricing clears demand in real time. One-click deploy turns procurement into minutes. Miner-friendly economics keeps capacity online. Takeaway: utilization beats gatekeeping.
SMB AI does not need peak-FLOPS theater. It needs stable, cheap, instantly rentable compute. Build where quota queues end. $LooPIN
this is a great analysis! a missing piece of "the H100 index" might be better benchmarking, since that rate could easily span a 2-3x spread across config, interconnect, uptime, and term at any moment. And the market is early: supply sits with ~10-20 large providers, so the reference price is thin and concentrated enough that a few sellers could move the settlement print. An independent, manipulation-resistant index has to exist before the derivatives can.
Cash-Settled Compute Futures: Mechanics of a Neocloud Short Hedge
The American Innovation Exchange’s GPU compute futures will soon be live for US investors, pending regulatory review. This post walks through a futures trade from the perspective of a neocloud hedging forward unsold capacity.
1. Background
Below is a single fully-worked numerical trading example from the perspective of a neocloud operator using a short futures position to lock in the forward sale price of capacity it expects to have available but has not yet contracted to end customers. The example illustrates position sizing, daily mark-to-market, variation margin, cash settlement under two states of the world, and the residual risks that remain after hedging.
The futures contract used in this example is cash-settled. At expiration, the position’s P&L is marked to the settlement value derived from the reference index. Cash settlement is the standard convention for commodity derivatives whose underlying is a service or a non-storable, non-deliverable quantity, and compute time falls into this category.
Architect will also offer ComputeConnect, an exchange-for-physical mechanism for converting positions into actual compute capacity. In a future post, we will detail how a cash-settled future can be converted to actual compute capacity using EFPs.
2. Contract Specification
The following specification is an example of how a single-accelerator, cash-settled compute future could be constructed. The actual exact contract unit, tick size, and margin parameters of any listed contract are set by the futures exchange and its clearing house.
• Underlying: NVIDIA H100 rental-price index
• Price quotation: U.S. dollars per GPU-hour
• Contract unit: 10,000 GPU-hours
• Minimum price fluctuation (tick): $0.001 per GPU-hour = $10.00 per contract
• Contract months: Monthly, extending along the forward curve
• Settlement method: Cash settlement against the final settlement value of the reference index
• Final settlement value: Published index level for the delivery period on the last trading day
• Initial margin: $6,000 per contract
• Maintenance margin: $4,800 per contract
3. The Hedging Problem
A neocloud is, in commodity terms, a producer: it holds an inventory of accelerators and sells their output, compute time, to customers. Its economic exposure resembles that of any producer holding unsold inventory. If the market rental rate for H100 capacity declines before the operator has contracted its available GPU-hours, the revenue realized on that capacity falls. The operator is therefore long the physical commodity and bears the risk of a price decline.
The standard remedy is a short hedge: the producer sells futures in a quantity that approximates its unsold physical exposure. A decline in the market rate reduces physical revenue but produces an offsetting gain on the short futures, because a short position profits when the futures price falls. A rise in the market rate does the reverse. In both directions the combined outcome converges toward a price fixed at the outset, converting an uncertain forward revenue into a substantially known one. This is the mechanism by which a producer "locks in" a forward sale price without having yet found a counterparty for the physical product.
When executed properly, a compute futures hedge removes price variance from a defined block of capacity so that build-out, financing, and margin commitments can be underwritten against a known revenue figure.
4. Establishing the Position
Assume the following facts as of the trade date.
• The operator projects that it will have 500,000 H100 GPU-hours of uncontracted, saleable capacity during the March 2027 delivery window.
• The March 2027 H100 future is trading at a price of F₀ = $2.40 per GPU-hour.
• The operator wishes to fix the revenue on the full 500,000 GPU-hours at the prevailing forward level.
Number of contracts. The hedge quantity is the physical exposure divided by the contract unit:
N = 500,000 GPU-hours ÷ 10,000 GPU-hours per contract = 50 contracts
Action. The operator sells (goes short) 50 March 2027 H100 futures at $2.40 per GPU-hour.
Notional value hedged.
50 × 10,000 × $2.40 = $1,200,000
Initial margin posted. Each contract has a value of 10,000 × $2.40 = $24,000 at the entry price. Initial margin is 25% of contract value:
50 × (0.25 × $24,000) = 50 × $6,000 = $300,000
The $300,000 is a good-faith performance bond held with the clearing broker (FCM), not a payment for the contracts. It is returned when the position is closed, adjusted for accumulated gains and losses. The economic significance of the position is the $1,200,000 of forward revenue whose price has now been fixed, posted against 25% of that sum in initial margin, which illustrates the capital efficiency of a marginable hedge relative to pre-selling the capacity outright.
5. Daily Mark-to-Market and Variation Margin
Futures positions are marked to market at the close of each trading session. The change in the settlement price is converted into a cash flow called variation margin that is debited from or credited to the position holder's account daily. For a short position, a fall in the futures price produces a credit and a rise produces a debit. The following two sessions illustrate the mechanism.
Session 1. The futures settle at $2.34, a decline of $0.06 from the entry price.
• Variation margin: ($2.40 − $2.34) × 500,000 = +$30,000
The account equity rises from $300,000 to $330,000. Because equity now exceeds the initial margin requirement, the $30,000 excess is available for withdrawal.
Session 2. The futures settle at $2.60, a rise of $0.26 from the prior close.
• Variation margin: ($2.34 − $2.60) × 500,000 = −$130,000
The account equity falls from $330,000 to $200,000. The maintenance margin requirement is 50 × $4,800 = $240,000. Because equity ($200,000) has fallen below the maintenance level ($240,000), the FCM issues a margin call requiring the operator to restore equity to the initial margin level of $300,000 with a payment of at least $100,000.
6. Cash Settlement at Expiration
On the last trading day the contract is settled in cash against the final settlement value of the reference index for the March 2027 window, denoted Sₜ. The cumulative profit or loss on the short futures position is:
Futures P&L = (F₀ − Sₜ) × 500,000
This figure is the algebraic sum of all daily variation-margin flows over the life of the position; the final session's mark simply brings the futures price into convergence with the settlement index. In parallel, the operator sells its 500,000 uncontracted GPU-hours into the physical market at the prevailing rate, which for this illustration is taken to equal the settlement index. Two scenarios are considered:
Scenario A — The rental rate declines (Sₜ = $2.00)
• Physical revenue: 500,000 × $2.00 = $1,000,000
• Futures P&L: ($2.40 − $2.00) × 500,000 = +$200,000
• Combined proceeds: $1,200,000
• Effective realized price: $1,200,000 ÷ 500,000 = $2.40 / GPU-hour
The physical revenue is $200,000 below the amount the operator would have received at the entry price, but the short futures position gains exactly $200,000, restoring the combined proceeds to $1,200,000.
Scenario B — The rental rate rises (Sₜ = $2.80)
• Physical revenue: 500,000 × $2.80 = $1,400,000
• Futures P&L: ($2.40 − $2.80) × 500,000 = −$200,000
• Combined proceeds: $1,200,000
• Effective realized price: $1,200,000 ÷ 500,000 = $2.40 / GPU-hour
The physical revenue is $200,000 above the entry-price benchmark, but the short futures position loses $200,000, again returning the combined proceeds to $1,200,000.
In both scenarios the effective realized price is $2.40 per GPU-hour, equal to the futures price at which the hedge was established. This symmetry is the defining characteristic of a fully executed short hedge: the operator has exchanged all upside above $2.40 for complete protection below it, fixing the forward revenue on the hedged block of capacity regardless of the direction of prices.
7. Hedge Effectiveness and Residual Risks
The example above assumes a perfect hedge, in which the price realized on the physical capacity equals the settlement index, and the hedged quantity equals the quantity ultimately sold. In practice, two residual exposures remain.
Basis risk. The rate the operator actually realizes on its own capacity, Rₜ, need not equal the settlement index level, Sₜ, because the index aggregates transactions across many configurations, geographies, and counterparties. The effective realized price generalizes to:
Effective price = Rₜ + (F₀ − Sₜ) = F₀ + (Rₜ − Sₜ)
where the term (Rₜ − Sₜ) is the basis. If, for instance, the operator realizes $1.95 per GPU-hour while the index settles at $2.00, the effective price becomes $2.40 + ($1.95 − $2.00) = $2.35 rather than the intended $2.40. Basis risk is the price of standardization: a single index cannot perfectly track the heterogeneous rate a specific operator obtains, and the residual is borne by the hedger. Selecting the reference SKU and delivery window that most closely matches typical physical exposure minimizes, but does not eliminate, this term.
Volumetric risk. The contract count is fixed at inception on the basis of a projection of saleable capacity. If the operator ultimately has only 450,000 saleable GPU-hours rather than 500,000, it is over-hedged by 50,000 GPU-hours, or five contracts, and that slice of the short position is no longer offset by any physical inventory. It becomes an outright short exposure to the compute price and produces an unhedged gain or loss. Conservative sizing, i.e. hedging a quantity at or below the highly probable minimum of expected saleable capacity, limits this exposure.
Liquidity and margin risk. As shown in Section 5, an adverse move in the futures price requires variation margin to be posted in cash before the offsetting physical gain is realized. A hedge that is economically sound can still impose a financing burden during its life, and this must be provisioned for.
8. Summary
A cash-settled compute future allows a neocloud to fix the forward sale price of capacity it expects to have available but has not yet contracted. By selling a number of futures equal to its uncontracted GPU-hours divided by the contract unit, the operator establishes a short position whose daily and terminal profit-and-loss offsets the change in the value of its physical inventory. In the worked example, 50 March 2027 H100 contracts sold at $2.40 per GPU-hour fix the proceeds on 500,000 GPU-hours at $1,200,000, whether the rental rate subsequently falls to $2.00 or rises to $2.80. The instrument converts an uncertain forward revenue into a substantially known one, subject to basis risk, volumetric risk, and the intraperiod liquidity demands of the margin system. For an operator underwriting build-out and financing commitments against future capacity sales, that conversion of variable revenue into a hedged forward curve is the central economic function of the trade.
Crusoe started off doing bitcoin mining, then successfully pivoted to AI, selling off their mining arm while it was still profitable
right now the demand for datacenters is so crazy that it only makes sense for new power allocations to go towards ML workloads
8 years ago, Crusoe set out to build the energy, data centers, and cloud infrastructure the AI era would need.
Today that infrastructure runs some of the most demanding AI workloads in the world.
Thank you to everyone building this with us. Here's to the next 8. 🎆
I first got interested in compute infra through the somewhat widely circulated details from meta’s llama 3 training run (419 unexpected failures over 54 days, roughly one every three hours, with 58.7% attributed to gpu-related issues - https://t.co/eC9aIfZZsq)
meta actually also has another paper (https://t.co/NqhtTquPrX) on how they operate these research clusters. the philosophy, as they put it, is “no second job failure from a bad node" (ambitious, but the right kind of optimism to deal with gpu reliability!!)
more recently beyond the research frontier, have been seeing more companies publish blogs on how they manage these systems. databricks released a post earlier this month on gpu reliability across customer workloads (https://t.co/QDxfHC9nPC), and cloudflare wrote about the inference side: prefill/decode disaggregation, kv-cache movement, token-aware routing, and multi-gpu execution.
also a side note for any student interested in this space, I'd recommend reaching out to the research computing team at your school! they often manage surprisingly large clusters and see many of the same failure modes as industry. personally, I've learned a lot from the hpc teams at stanford, and you can often learn operational details you’d never get from a blog post, from where they procure and configure their gpu systems, including vendors like colfax
Last Seen Users on Sotwe
Kasırgalı
Seen from Turkey
Tantek
Seen from Austria
Cum Banger
Seen from Turkey
songuyen
Seen from Vietnam
something
Seen from Indonesia
divulgações boy
Seen from Brazil
LockerRoomGuys
Seen from Mexico
Mom_desire💦
Seen from Singapore
hijab
Seen from Pakistan
Penikmat bra dan cd untuk coli
Seen from Singapore
Most Popular Users
Elon Musk
@elonmusk
240.9M followers
Barack Obama
@barackobama
119.2M followers
Donald J. Trump
@realdonaldtrump
111.8M followers
Cristiano Ronaldo
@cristiano
111.5M followers
Narendra Modi
@narendramodi
107.1M followers
Rihanna
@rihanna
97.9M followers
NASA
@nasa
92.2M followers
Justin Bieber
@justinbieber
91.1M followers
KATY PERRY
@katyperry
88.2M followers
Taylor Swift
@taylorswift13
82M followers
Lady Gaga
@ladygaga
73.6M followers
Virat Kohli
@imvkohli
70.7M followers
Kim Kardashian
@kimkardashian
70M followers
YouTube
@youtube
68.7M followers
Bill Gates
@billgates
64.2M followers
Neymar Jr
@neymarjr
63.5M followers
The Ellen Show
@theellenshow
62.4M followers
CNN
@cnn
61.9M followers
Selena Gomez
@selenagomez
61.2M followers
X
@x
60.8M followers