Sanketh

Technology, strategy, investing etc etc etc

[Sanketh @5anketh]

Kimi3/GLM5.2 points to a future where software engineers(if they are still around) will have harness that uses SOTA for planning while using local models for workers.

[Sanketh @5anketh]

How operating margins improve for cloud businesses as scale grows through better utilization and efficiency win

[Sanketh @5anketh]

@mukund 37 and accelerating while being supply constrained

[Gal Zahavi @galdawave]

84% of our Anthropic API cost came from cache reads and writes.

Not input tokens, not output tokens.

Cache.

At Google, I honestly never thought much about the cost of AI. Joining an early stage startup completely changed that. Every dollar matters, so I became pretty obsessed with understanding where our token usage was actually going.

Most of our usage made sense. Our team averages around 70 to 100 PRs a day, so a large token bill is expected.

Then Fable was released and something felt very wrong. We were producing fewer PRs, but our cost was nearly 20x higher, and yes, I know what you’re thinking. Fable 5 is much more expensive than opus, but even after accounting for that, the math still didn’t work.

So I spent a week tracking everything claude code reported and comparing it against the Anthropic platform. The numbers mostly matched.

The surprising part was where the money was going. 84% of our usage came from cache reads and writes, split almost evenly, with cache reads slightly ahead.

In theory, caching should make things cheaper. In long running sessions, especially when claude code is acting as the main orchestrator, it became the biggest part of our bill.

Compaction helped, but only briefly. One request later, cache reads and writes would jump again. So I changed the workflow.

I now use Fable for planning, create the Linear ticket, then start a completely fresh session with Sonnet and let Codex handle the implementation. That simple change reduced our cost by around 50%.

Biggest takeaway:

If you’re paying for API usage, don’t just watch input and output tokens. Watch cache reads and writes very carefully.

Honestly, I suspect something similar happens on Claude subscriptions too. You just never see the bill.

[Oguz Erkan @oguzerkan]

Hyperscalers will print cash.

Morgan Stanley predicts ~90% net margin in token sales from $NVDA Feynman Data Center.

Some hyperscalers like $AMZN and $GOOG use custom ASICs and reportedly achieve even higher inference margins.

Couldn’t be more bullish.

$AMZN $GOOG $MSFT $META

[tae kim @firstadopter]

Since many of you seem to have the memory of a goldfish, here's how the hyperscaler capex → free cash flow cycle works as explained by Andy Jassy back in April. He's right.

Amazon: "We have customer commitments that make our capex investments predictable. We’re not investing approximately $200 billion in capex in 2026 on a hunch. The recent OpenAI commitment (over $100 billion) is an example of this, but there are several other customer agreements completed (and unannounced), or deep in process. Of the AWS capex we expect to spend in 2026, much of which will be monetized in 2027-2028, we already have customer commitments for a substantial portion of it.

We are willing to make large capex investments and endure short-term FCF headwinds for the substantial medium to long-term FCF surplus.

AI is a once-in-a-lifetime opportunity where the current growth is unprecedented and the future growth even bigger. AWS has a significant leadership position with the broadest functionality, strongest security and operational performance, largest share of customers and revenue, strong desire from customers to run their AI in AWS, and an opportunity to build what could be a new pillar for Amazon in chips.

We’re not going to be conservative in how we play this—we’re investing to be the meaningful leader, and our future business, operating income, and FCF will be much larger because of it."

[Sanketh @5anketh]

How operating margins improve for cloud businesses as scale grows through better utilization and efficiency win

[Sanketh @5anketh]

@borrowed_ideas With 82% and accelerating growth, should be faster than 2030?

[Sanketh @5anketh]

@charliebilello I am not sure why street is expecting very high efficiency so early in the investment cycle. I am surprised that it is positive at this scale.

[Sanketh @5anketh]

@charliebilello I am not sure why street is expecting very high efficiency so early in the investment cycle. I am surprised that it is positive at this scale.

[Ben Bajarin @BenBajarin]

I'm realizing, as shocking as it sounds, the broader market/industry still isn't grasping the inflection point that is inference at scale.

That and we are still EARLY innings rack scale compute architectures. Which are required for inference...

[Chetan Puttagunta @chetanp]

Google Cloud is now at ~$100B revenue run-rate, growing 80%+ and is 40% of Google's search business. Wild that Google Cloud could be nearly as big as Google Search in revenue in the next year or two on its current trajectory.

[unusual_whales @unusual_whales]

Coinbase, $COIN, has successfully cut its AI spending by nearly 50% while simultaneously increasing its token usage, per YF

[Sanketh @5anketh]

Kimi3/GLM5.2 points to a future where software engineers(if they are still around) will have harness that uses SOTA for planning while using local models for workers.

[Cursor @cursor_ai]

We had a team of agents rebuild SQLite from its 835-page manual.

It created a replica in Rust which passed 100% of a held-out test suite.

Interestingly, cost varied 15x depending on which model mix we used.

[Sanketh @5anketh]

@oguzerkan Agree! Open models are catching up or already on par with Opus 4.8 and GPT 5.5.

These models are good enough in capability to unlock substantial adoption for economic tasks

[Sanketh @5anketh]

@oguzerkan Agree! Open models are catching up or already on par with Opus 4.8 and GPT 5.5.

These models are good enough in capability to unlock substantial adoption for economic tasks

[Sanketh @5anketh]

Underreported number that shows that models can have ~70% gross margin. Bullish on ROIC for hyper scalers

[Jing Yang @jingyanghk]

And we just updated our story with new reporting that DeepSeek's gross margin from selling v4 API access is...70% to 80% (!)

[Sanketh @5anketh]

@GavinSBaker Open weights models are already good enough to unlock large blocks of economic value.

[Sanketh @5anketh]

@GavinSBaker Open weights models are already good enough to unlock large blocks of economic value.

[Sanketh @5anketh]

The AI trade may be shifting. Wall Street is starting to rotate from the companies building AI to the companies that can make money from it.

Software stocks are showing that AI may not wipe them out after all, while semiconductor stocks may be near their peak.