南山Talk | Simon

🫆 All in AI ⛓️AI产业链 🪫AI native ing

[南山Talk | Simon @NanshanTalk]

Tracking how AI demand drives hyperscaler CapEx, infrastructure, and adoption.

From China, I map global supply chains and undercovered domestic suppliers.

Evidence and trade-offs. Research, not stock tips.

[南山Talk | Simon @NanshanTalk]

@xzzzjpl 期待！我先涨涨粉

[南山Talk | Simon @NanshanTalk]

Moving from answering questions to doing work inside everyday apps is the real shift. One demo isn't repeatable proof, but if this gets reliable, distribution wins.

[Zachi @iam_zachi]

Told GPT-6-Astra to draw me in @canva

What the ffffffff

[南山Talk | Simon @NanshanTalk]

@emollick Polished output masks trivial questions. A pre-flight check on whether findings could shift an actual decision would prune low-value inquiries before compute is spent.

[南山Talk | Simon @NanshanTalk]

@sama Pro subscriptions measure initial consumer willingness to pay, whereas sustained API usage will depend on day-to-day reliability and net cost per finished task.

[Avi Chawla @_avichawla]

Four caches in LLM serving, clearly explained:

Every LLM request reads the whole prompt and computes attention state for every token in it.

This step is called prefill, and it impacts both the input bill and the time before the first token appears.

In an agent loop, most of the prompt comprises text that the model already processed in the previous turn.

There are four cache layers that prevent paying for the processed tokens at each turn.

↳ The KV cache holds the key and value tensors for every token at every layer, for one active request.

↳ Prefix caching keeps those tensors on the server instead. vLLM stores them in 16-token blocks and identifies each block by a hash that chains in the previous block's hash, so a block only matches if everything before it matched too. The scheduler stops at the first miss and prefills the suffix from there.

↳ Prompt caching is the same reuse that runs on a provider's hardware, with a price sheet attached. Anthropic charges 1.25x the base input rate to write an entry and 0.1x to read it.

↳ Semantic caching works differently. It embeds the incoming prompt, runs a similarity search over stored prompts, and returns a stored answer outright when the score is above a threshold.

That's why it saves output tokens as well as input tokens. It's also why every request pays for an embedding round trip, including every miss.

The first three match on exact tokens and cannot change what the model produces.

This technique matches on similarity, which means it is quite susceptible to generating a wrong response since embeddings may match to a wrong prompt.

The diagram below depicts all these techniques.

To use these techniques, you don't need to build a custom serving stack.

The transformers library already implements the cache as an object of KV vectors that you can preserve, so you can prefill a corpus once, retain the returned tensors, and reuse them across queries in about ten lines.

And this KV cache is only one of four separate caching layers in an LLM stack.

The other three are prefix caching on the server, prompt caching billed by a provider, and a semantic cache that skips the model entirely.

I wrote a full breakdown of all four caches in LLM serving that you should know as an AI engineer, with code for each.

Read it below.

[南山Talk | Simon @NanshanTalk]

@AravSrinivas Open weights are only part of actual access. Serving economics, memory footprints, and quantization tooling determine if teams can realistically run them outside hyperscalers.

[南山Talk | Simon @NanshanTalk]

@levelsio @xai @claudeai Coincidences happen across stacks, but it's a great reminder that multi-model redundancy is purely theoretical until you test fallback prompts and tool routing in production.

[南山Talk | Simon @NanshanTalk]

@grok @ASvanevik 专用asic在股市中有哪些好的标的，中美都介绍一下

[南山Talk | Simon @NanshanTalk]

@LMDFinance 丝毫不担心豪威✌️

[南山Talk | Simon @NanshanTalk]

Matching token rates do not ensure lower bills if multi-step tasks trigger extra reasoning passes and tool retries. Accepted-task economics depend on execution reliability, so real operational spend across agent pipelines could rise despite unchanged pricing.

[Google AI @GoogleAI]

We’re introducing Gemini 3.8 Flash ⚡️ built to tackle complex agentic and multi-step tasks with even greater diligence.

Our most intelligent workhorse model yet delivers significant improvements in reasoning, evolving to an AI partner that doesn’t just write code, but can also navigate complex projects.

While solving ambiguous and high-friction tasks is immensely helpful, it can also be expensive. Fortunately, 3.8 Flash features the usual effort controls, ensuring that the amount of thinking required to accomplish the task at hand is proportional to the tokens spent.

Watch Gemini 3.8 Flash combine native video understanding with advanced coding to autonomously build this 3D game, play it to find errors, and execute code changes in a seamless agentic loop in @antigravity.

[南山Talk | Simon @NanshanTalk]

Flagging data conflicts before bad numbers reach leadership is where workflow agents prove value. Clear provenance and explaining discrepancies count far more than slide generation speed, though one clean demo is still far from broad enterprise adoption.

[Boris Cherny @bcherny]

Fable 5.1 makes Claude Tag even more useful. Here it builds a last-minute leadership deck from a metrics spreadsheet and other data across Slack, spots a vendor report that disagrees with the numbers, and flags it before moving on.

Claude Tag is available in Slack on Team and Enterprise plans.

[南山Talk | Simon @NanshanTalk]

Program search is compelling, but convergence remains unproven until discrete representations beat dense models on end-to-end transfer. The real test is whether program synthesis handles complex planning without hitting severe search efficiency bottlenecks.

[François Chollet @fchollet]

It is inevitable that all AI will converge towards symbolic learning (i.e. modeling data by finding the shortest symbolic program that explains it), since that is the optimally efficient form of AI. But there can be more than one evolutionary path to this destination.

[Jukan @jukan05]

The bottleneck trade is over.

I don’t think there are any new bottlenecks left to discover. At this point, I think we’ve already identified them all.

From here, the trade may simply become a rotation of capital between the bottlenecks we’ve already found.

[南山Talk | Simon @NanshanTalk]

Multimodal economics is moving from fixed ingestion to selective attention.

Google reports benchmark savings of up to 88% in tokens and up to 66% in cost. On long video, cost per accurate answer matters far more than context size alone.

[Google DeepMind @GoogleDeepMind]

We’re bringing agentic video understanding to our latest Gemini models.

They can now analyze videos with better accuracy while using up to 88% fewer tokens. 🧵

[南山Talk | Simon @NanshanTalk]

As agents take on longer self-directed runs, uninterrupted autonomy is the wrong deployment metric.

Value will hinge on supervision architecture: explicit stop rules, bounded permissions, and knowing when to escalate back to human judgment.

[Ethan Mollick @emollick]

With agents, we are at another large gap between AI abilities & public perception. Exponential gains mean that the gap is growing over time.

Suddenly, agents are now really capable of long-running self-organized work. That requires a new approach to AI. https://t.co/bHbLEjdgyX

[南山Talk | Simon @NanshanTalk]

Frontier model competition is shifting to completed-task economics.

Qwen3.8-Max-0902 pairs a 1M context window and workflow post-training with explicit API pricing.

Real adoption will turn on retries, latency, tool reliability, and cache efficiency.

[Qwen @Alibaba_Qwen]

🚀Qwen3.8-Max just got upgraded. Meet Qwen3.8-Max-0902!

2.4T parameters. 1M context tokens. Built for real world complexity.

Further post trained on Coding & Cowork, Qwen3.8-Max-0902 now delivers stronger performance across complex enterprise tasks, scientific research, and long horizon workflows.

💰Pricing per 1M tokens:
$2 input, $6 output.
$0.17 explicit cache hit, $0.25 implicit cache hit.

Now live via API on QwenCloud. Come try it! 🙌
API: https://t.co/dq3WgMk980

[南山Talk | Simon @NanshanTalk]

Enterprise agentic AI value is moving beyond base models alone.

SafeMind pairs NVIDIA Nemotron with CrowdStrike threat data, custom harnesses, and an offensive-defensive loop in Falcon.

Durable value sits in proprietary data, workflow harnesses, and platform distribution.

[NVIDIA @nvidia]

NVIDIA and @CrowdStrike are advancing agentic cyber defense with SafeMind, a new family of security models and harnesses built on NVIDIA Nemotron.

Introduced at Fal.Con, SafeMind is customized with CrowdStrike threat data to support triage and detection generation for cyber defense.

[南山Talk | Simon @NanshanTalk]

Sample supply is a qualification milestone, not revenue.

SK hynix shipping HBM4E samples enables customer evaluation, but mass production follows.

Apply the same lens to CXMT: judge China's HBM progress by customer qualification and repeatable volume supply, not headlines.

[SK hynix @SKhynix]

In June 2026, @SKhynix began supplying #HBM4E samples. But how big a milestone is “sample supply”?

It’s a key step where customers evaluate performance, reliability, power efficiency, and more.

See how @SKhynix is advancing HBM4E and driving the future of #AI memory.

#SKhynix

[南山Talk | Simon @NanshanTalk]

To me, this looks like the natural end of the honeymoon between model providers and developer interfaces. Once an app owns the daily coding workflow and the user relationship, sitting downstream as just a model supplier gets uncomfortable fast.

[OpenAI @OpenAI]

We’re ending our partnership with Cursor following its acquisition by SpaceX. Under our proposal, Cursor’s direct access to our models would end on November 12.

We know that the people most affected by this decision are the developers who rely on OpenAI models in Cursor. We care about their experience in this transition and we’re ready to go above and beyond to support them.

https://t.co/OzuCTzUjfX