SemiAnalysis

[SemiAnalysis @SemiAnalysis_]

The larger cost reduction opportunity lies further out with larger interposers; for example, 9.5x reticle size interposers yield 9 per panel versus 4 per wafer. The transition to larger reticle sizes and larger panels depends on equipment maturity. (2/2)

[SemiAnalysis @SemiAnalysis_]

ASE showcased its panel-level packaging (PLP) solution for AMD’s Venice CPU, with initial production targeted for Q1 2027, likely at limited scale at its Kaohsiung facility. ASE and its subsidiary SPIL have worked with AMD to implement the second generation of their Fan-Out Embedded Bridge (FOEB) technology in Venice. The current 310 × 310 mm panel accommodates 25 Venice interposers after edge loss, compared with 22 on a traditional round 300 mm carrier wafer, roughly a 14% increase in gross package count. This modest improvement in package output per carrier represents a starting point for panel-level interposers with Venice’s 2.7x reticle size, rather than evidence of a dramatic reduction in cost. (1/2)🧵
Source: OCP ASE Booth

[SemiAnalysis @SemiAnalysis_]

At a small number of concurrent sessions, CPU DRAM offloading is not needed since all the sessions' KVCache (even for idle sessions waiting for tool-use calls to finish) can be stored in HBM. (2/2)

[SemiAnalysis @SemiAnalysis_]

ALERT🚨: The number of tokens that can be cached can increase by 20-40%🚀 by enabling DRAM KVCache offloading optimization at 12-16 concurrent agentic sessions. A B200 only has 180 GB of HBM, and after weight storage, only 100 GB of HBM per chip remains for KVCache.

THIS IS NOT ENOUGH FOR LONG-CONTEXT AGENTIC INFERENCE WORKLOADS. Thus, with SGLang HiCache, you can also offload KVCache to the CPU's DDR5/LPDDR5X memory, which helps improve performance. (1/2)🧵

[SemiAnalysis @SemiAnalysis_]

PoV of Jensen after reading TPUs are coming to Oracle Cloud
https://t.co/7vFiRSX7dd

[Oracle Cloud @OracleCloud]

We’re deepening our collaboration with @googlecloud to give customers more options for building and running AI at scale.

Google TPUs are coming to Oracle Cloud Infrastructure (OCI), marking the first time Google TPUs will be available on a cloud platform outside Google. Customers will have more flexibility to match AI infrastructure to demanding training and inference workloads, optimizing their environments for performance, throughput, and economics.

Learn more: https://t.co/YWN9P3xp7y #AIWorld

[SemiAnalysis @SemiAnalysis_]

Spotify:https://t.co/uMhoocdpzJ

[SemiAnalysis @SemiAnalysis_]

Anthropic’s S-1 shows inference revenue per megawatt nearly quadrupling in a year.

“Inference revenue per megawatt went from sixteen million last year to sixty plus currently, across the entire fleet. AI model businesses scale extremely well.”

“Revenue grew 1,368% year over year for the three months ended June 30, compared to compute and infrastructure expenses that grew 300% during the same period.”

“How much of that is Anthropic being geniuses versus market-wide efficiency? We’re seeing GB be 50x better than Hopper.”

“They have TPU, Trainium, NVIDIA GPU and AMD GPU all online. New hardware is always more performant, but you need the software to use it. Who’s got the best software? Probably the company with the top engineers and the best model.”

[SemiAnalysis @SemiAnalysis_]

Watch Now: https://t.co/wym6QWlK0j

[TBPN @tbpn]

The @SemiAnalysis_ team gives two examples why we're still so early with AI adoption outside of the tech bubble:

1. $1,500/month token budgets at Fortune 500 tech companies:

SemiAnalysis Tokenomics co-lead @SaasquatchC explains that a lot of engineers don't even hit that budget at these big, older companies, and that traditional back-office work is lagging behind, as well.

2. The "AI-pilled" person at a wedding using Microsoft Copilot:

SemiAnalysis President @fabknowledge went to a wedding this weekend in Fort Worth, Texas.

He met someone who claimed to be AI-pilled, so of course he wanted to talk to Doug about AI.

He told Doug the technology is crazy, and he couldn't believe the rest of the world wasn't talking about it.

When Doug asked what he used, he said Microsoft Copilot.

[SemiAnalysis @SemiAnalysis_]

BREAKING NEWS🚨: NVIDIA just submitted its first Rubin NVL72 Kimi K3 submission, and it can reach over 850 tok/s/user, even at 2.8 Trillion parameters, with over 3x higher throughput on @vllm_project!

Great work by the NVIDIA team, and excited for Rubin Software 1.0 GA coming soon!

[SemiAnalysis @SemiAnalysis_]

Spotify:https://t.co/CPDADvr1Ou

[SemiAnalysis @SemiAnalysis_]

@fabknowledge @JeremieEO @SaasquatchC @JordanNanos Watch Now: https://t.co/giYA81vb4S

[SemiAnalysis @SemiAnalysis_]

@fabknowledge @JeremieEO @SaasquatchC @JordanNanos Spotify: https://t.co/rUpd1FEv1t

[SemiAnalysis @SemiAnalysis_]

To find out more, subscribe to our Accelerator & HBM Model (2/2) https://t.co/iNX6TOZ3hd

[SemiAnalysis @SemiAnalysis_]

Alibaba's Panjiu 128 is not a true rack scale up. Panjiu 128 is made up of two 64 scale up size systems with ALink switch cartridges in the back. Alibaba's own architecture write-up says a single 128 domain would need 128-port switch chips, and the ICN Switch 1.0 is a 25.6 Tbps part sized for 64. We covered this during our Alibaba Apsara Conference takeaways note where we found that only Panjiu 144, with switches in the middle of the rack, is a true rack-scale domain which is not due until 2028. (1/2)🧵

[SemiAnalysis @SemiAnalysis_]

Read more about their low latency kernels in their blog (2/2) https://t.co/DgiugFyThH

[SemiAnalysis @SemiAnalysis_]

Congrats to the SGLang and RadixArk team for their amazing work on MegaKernel + PD, improving both high concurrency throughput and low-latency serving as well! (1/2)🧵

[SemiAnalysis @SemiAnalysis_]

The most mind blowing stat we've seen so far is this:

"As of June 30, 2026, we had raised an aggregate of $141.1 billion of capital and had cash and cash equivalents of $112.4 billion and short-term investments of $19.3 billion."

That means to date, they have burned 9.7b, they are growing at a rapid pace, and are profitable on an operating income basis as of the June quarter! This is likely the most efficient scaling of a business in the history of capitalism.
https://t.co/bWYbqKOMDN

[Anthropic @AnthropicAI]

Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.

Read more: https://t.co/ofhv0DK3Zp

Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.

[SemiAnalysis @SemiAnalysis_]

PROFIT ALERT🚨: AMD @vllm_project can generate up to $3/hr/chip in pure profit serving the open-source Kimi K3 model, even after accounting for the rapid increase in chip rental prices post-Claude Code.

Kimi K3 2.8T is a great model for AMD, as it has over 2.8 trillion parameters that can take advantage of AMD's 288GB per chip. B200 CANNOT fit Kimi K3 in a single server, so it has to use tensor parallelism or pipeline parallelism for communication across 400Gbit/s InfiniBand/Ethernet, which is 18x slower than the BW within an NVLink domain.