Antonio Alonso-Stepanov

compute! stanford cs

[Antonio Alonso-Stepanov @antonioalstep]

if you have a .edu, .gov or .mil email you can now get a full 8x B300 node for $4/GPU-hour. nebius charges $9.50 on demand and a lot of clouds are sold out

AMD MI355X nodes are $3 and the wait right now is under a minute

you just ask marshall, their AI agent, and it finds you a node

this is a major public good for researchers who've been priced out of frontier compute

thank you @AnjneyMidha

[Anjney Midha @AnjneyMidha]

hundreds of MI355x and B300 nodes

now live on https://t.co/t0uE0bekz3

subsidized for all .edu and .govs

it’s time to scale the frontier ecosystem

for the public benefit

[Antonio Alonso-Stepanov @antonioalstep]

modal is a good example of where compute markets are heading. they sell gpus by the second with no commitment for less than the big neoclouds charge on demand

-> b200 is $6.25/hr on modal vs $8.50 at nebius
-> b300 is $7.10 vs $9.50
-> h200 is $4.54 vs $5.40

they rent from 20+ clouds and commit to capacity ahead of demand. pooling demand across all their customers keeps utilization high, which is what lets them price lower

the more compute trades like a commodity, the more value goes to whoever aggregates supply and matches it to demand

[Erik Bernhardsson @bernhardsson]

https://t.co/nByTwj2JPX

[Antonio Alonso-Stepanov @antonioalstep]

you can get kimi k3 on liquid inference for $0.39 per million input tokens and $0.57 output right now

moonshot's own api is $3 and $15. the cheapest provider on openrouter is $0.73 and $10

it's still cheaper after their cache discounts too

the difference is market design. on openrouter each provider lists a fixed price and openrouter spreads traffic across them. on liquid every request is an auction, providers can reprice any minute, and the cheapest offer that meets your speed and quality rules wins

markets will exist for compute and tokens just like for other commodities

[Brett Harrison @BrettHarrison]

Stats from Liquid Inference’s first few days live:
• Cheaper token prices than 98% of OpenRouter traffic
• ~2B tokens served
• 24 verified providers onboarded to the network
• 88% cache hit-rate
• 26 harnesses/IDEs supported

Buckle up, many new features launching this week.

[Antonio Alonso-Stepanov @antonioalstep]

@karpathy if you're in the 99.8% that's never experienced what a frontier long-running agent can do, it makes sense to think it's all a bubble.

there's a strong economic incentive to learn though. i hope we have enough compute when the time comes for everyone to truly adopt it

[Antonio Alonso-Stepanov @antonioalstep]

the token index drop is mostly an explosion in cheap model usage, not less frontier spend

on openrouter models under $0.25 per million tokens are now 3/4 of all tokens, and that shift alone cut the average price per token ~31% over the last month

frontier usage keeps growing. tokens on the top closed models grew 55% over the last month and they're still about half of all spend

and there will always be demand for frontier tokens. for r&d and anything adversarial like negotiation or trading, you want the best model

meanwhile b200 rental prices are up 67% since mid june

[Kuppy @hkuppy]

In a normal functioning ecosystem, when the price of tokens drops below the cost of production, the quantity of tokens declines to normalize with pricing. In this case, nothing in AI has any unit economics and you don’t see normalization because everyone is still going full-speed ahead with external funding. (Sort of like shale oil in 2016-2019). Meanwhile, compute prices are based on LT contracts that were signed at above market prices, and are now capital destructive. They’ll eventually catch down to token prices in free-fall.

What we’re seeing here is spot price negatively diverging from term pricing. Usually, when this happens in a cyclical industry, it’s INCREDIBLY BEARISH. Especially when there are take-or-pay contracts that make it difficult to ration supply at prices below cost. What we’re witnessing is a world where losses will accelerate, just as more supply comes online….

[Antonio Alonso-Stepanov @antonioalstep]

google knew the risk of cutting noam shazeer's compute (the stock fell by $100 billion the day his departure was announced) but did it anyway because they didn't have enough

dario personally calls ppl to beg for compute

imo ppl are still underestimating the importance and scarcity of compute

it's even harder for startups who can't fund long multiyear contracts or put 30% down

compute is the oxygen of ai

[Choblin @choblin29]

WSJ is reporting that Anthropic CEO Dario Amodei personally called Meta's Alexandr Wang seeking more compute. Meta considered providing chips, then declined.

Inside GOOGLE, Demis Hassabis has expressed frustration about Google not allocating enough compute to long-term research. Sergey Brin sometimes overrides the company's formal compute allocation process, redirecting resources toward projects he personally considers important.

Google also cut compute allocated to Noam Shazeer's team in favor of another project. Shortly afterward, he left for OpenAI.

[Antonio Alonso-Stepanov @antonioalstep]

agree, open tokens burn about as much compute as closed tokens. inference providers (like @baseten and @FireworksAI_HQ) have positive margins on top of their compute costs, even with dozens of companies competing to serve the same models on openrouter

there’s a question of why. in a world where AI can spin up an efficient inference stack for anyone, a large software edge shouldn’t last long

a lot of it is that inference providers signed long-term compute contracts making them effectively long compute. those contracts have gotten more valuable since they signed. a 1-year H100 contract went from ~$1.70 last october to ~$2.75 today, and B200 rentals are up ~85% since june

so part of that margin is effectively a compute trade rather than purely a software moat

[Gavin Baker @GavinSBaker]

Why open-weight models that compress margins at the model layer are net positive for AI infra demand, exhibit 1001.

An open-weight token consumes *roughly* the same amount of compute as a frontier token for a similar size model.

[Antonio Alonso-Stepanov @antonioalstep]

a GPU cluster can pass every benchmark and still go down for days

IREN’s response was NVIDIA exemplar status, which shows its cluster hits NVIDIA’s reference performance on their benchmark recipes. whether a site stays up month after month is a separate question

here’s what I’d ask before signing a GPU contract

- which site am I on, and is power, cooling and network N+1 or N+0
- does the provider run that datacenter or rent colo space and rely on the landlord’s techs
- what was uptime at that site over the last 90 days, and how does the SLA define downtime
- how fast do health checks catch a bad GPU and pull it out of your jobs, and are there hot spares
- on NVL72, what happens when a tray dies. it takes 4 GPUs out of the NVLink domain, so ask if they run the rack degraded or swap it and how fast
- which XIDs they act on. the ones that mean bad hardware are 79 (GPU fell off the bus), 48, 94 and 95 (memory errors) and NVLink errors (74 on hopper, 144 to 150 on blackwell). 13, 31 and 43 are usually your own code
- was the cluster burned in with the GPUs and network stressed at the same time
- link flaps and error counters on the infiniband or RoCE network

the providers with good answers usually love being asked

[SemiAnalysis @SemiAnalysis_]

IREN continues to come up in our conversations with neocloud customers, and usually for the wrong reasons. In Prince George and Mackenzie in Northern BC, Canada, we’ve heard the stories: multi-day power outages, network upgrades, storage failures, air quality controls leading to link flaps, XIDs everywhere. The #1 worst site in the industry, according to users.

And we are some of their users, as we’ve gotten to test IREN GPUs from multiple providers that resell their capacity. A lot of our users we are hearing from who aren’t having a good time aren’t renting from IREN resellers, but from IREN themselves. (1/3)🧵

[Antonio Alonso-Stepanov @antonioalstep]

as more and more of total compute is used for inference it makes sense to stand up compute as quickly as possible and modular is great for that; you can find power where it is and deploy prefabricated pods

crypto mining took this scrappy approach because cost of power mattered immensely

ai takes it to optimize how quickly you can stand up new capacity

[Antonio Alonso-Stepanov @antonioalstep]

some inference providers on openrouter are playing a funny game

openrouter's routing seems to reward whoever lists the cheapest input price so a few providers price input near zero and make it back on output

on deepseek v4 flash today, relace has the lowest input price ($0.01 per million tokens) and gets 33% of all traffic. but it charges $1.28 per million output tokens while streamlake charges $0.08

even though 94% of tokens on this model are input, relace still ends up almost 4x more expensive than streamlake. for reasoning-heavy work it's about 15x

in theory openrouter should be a really good marketplace where competition drives prices down fast. but that only works if the incentives reward the cheapest real price, not gaming the listings. it's a really important market design problem

[Antonio Alonso-Stepanov @antonioalstep]

the big AI clouds are building in really different ways right now

Google is unique in that it owns basically every layer. it designs its own TPU chips, builds and runs its own data center campuses, and owns the application layer with Gemini, Search, YouTube, etc. its cloud business runs a very healthy operating margin.

SpaceXAI is similarly very integrated. they've put their own gas turbines and Tesla Megapacks next to their data centers and run Grok on a lot of their compute, though they still buy every GPU from Nvidia.

Crusoe develops the sites and the power, although for its biggest sites like Abilene, Blue Owl owns most of the campus. Nebius owns its Finland site, designs its own servers, and fun fact, pipes the server heat into the local town!

CoreWeave takes a different approach and leases every data center, only owning the GPUs and software. Fireworks goes further and mostly rents GPUs from a bunch of providers to sell inference, though we're seeing more inference providers start buying their own compute to expand margins and bring on more capacity

the more of the stack you own, the more capital it takes upfront, but also the better you can work around bottlenecks, which are getting more pervasive as the compute shortage worsens

[Antonio Alonso-Stepanov @antonioalstep]

@FPuklowski that's much better than B300s at like $20m per megawatt a year. Is financing much harder w heterogeneous compute?

[Antonio Alonso-Stepanov @antonioalstep]

@jessiedong_ nvidia gpus are currently also a bit easier to finance since lenders know them better. cost of capital really matters for compute deployments

[Antonio Alonso-Stepanov @antonioalstep]

@nicbstme crazy to think about what will happen with the compute shortage as everyone starts actually using agents which take orders of magnitude more compute than chat, which most ppl still use outside of the bubble