Wei Shi

PTO:

[Linas Beliūnas @linasbeliunas]

Instead of watching 1 hour of Netflix tonight, watch this ex-Google Chief Scientist Jeff Dean’s lecture. It’s the clearest explanation I’ve seen of the full AI engineering stack - from building LLMs from scratch all the way to one human coordinating 100 agents.

The best part is that it’s useful whether you’ve never touched a model or you’ve been shipping agent systems every day for the past year.

Bookmark it & watch the whole lecture this weekend, because it might end up being the most valuable thing you learn all week.

[Wei Shi @weishi]

Scientist AI from Yoshua point of view

[Voice Arena @voicearena_ai]

Announcing the Voice Arena STT Leaderboard, our human-verified benchmark testing how accurately speech-to-text models transcribe real, spontaneous conversation across 7 languages

The leaderboard covers 🇺🇸 US English, 🇮🇳 Hindi, 🇧🇩 Bangla, 🇧🇷 Brazilian Portuguese, 🇻🇳 Vietnamese, 🇷🇴 Romanian and 🇸🇦 Arabic. Every model is evaluated on the same held-out corpus per language: real, unscripted phone calls between native speakers, recorded in-country on their own phones, with the noise, fillers and interruptions of natural conversation.

Unlike public STT benchmarks, results are measured on fully proprietary audio - no model has seen a second of it in training. @Reson8Offical's resonant-1 leads the US English board at 4.34% WER, ahead of models from @Microsoft , @Google , @OpenAI and @Meta . @SarvamAI leads in Hindi and Bangla, while Microsoft takes #1 in Romanian and Brazilian Portuguese.

Key elements of the Voice Arena STT Leaderboard:
➤ Real speech, real conditions: audio is drawn from unscripted phone calls between native speakers, recorded in-country on the speakers' own devices, preserving the background noise, fillers and interruptions of production traffic.
➤ 100% proprietary, held-out audio: the corpus is collected and owned by Voice Arena. No model has seen a second of it in training, so results cannot be inflated by dataset contamination.
➤ Human-verified ground truth: transcripts are produced by native transcribers through a six-stage pipeline, and only segments verified 3-of-3 survive into the final test set.
➤ Corpus-level WER with symmetric, language-aware normalization: the same normalization is applied to both references and hypotheses, so models are penalized for recognition errors, never for spelling choices.

Key results for the Voice Arena STT Leaderboard:
➤ The most accurate US English model is not from a big lab: Reson8's resonant-1 leads at 4.34% WER, ahead of Microsoft, Google, OpenAI and Meta.
➤ Model choice alone is worth ~44%: on the US English board, #1 sits at 4.34% WER and #20 at 7.74%. Same audio, same test set. Picking the right model cuts nearly half of your transcription errors before you touch anything else in the pipeline.
➤ There is no global winner: the #1 model changes with the language, with Reson8 leading US English, Sarvam leading Hindi and Bangla, and Microsoft leading Romanian and Brazilian Portuguese. Teams building for more than one market will not be covered by a single vendor at the top of every board.

[Bing Xu @bingxu_]

We think super long context is a problem solved by multi-agents. Recently we applied SwarmOS to research a problem we are interested in: What is China AI-stack’s pressure path. We shared the full generated report in this blog as well.

[Madan Khadka @MadanKhd]

Doom scrolling led me to an unexpected gem: Understanding Latency Hiding on GPU by Vasily Volkov (UC Berkeley). It explains many GPU architecture and performance concepts that even Programming Massively Parallel Processors doesn't cover in depth.

[diva @divaagurlxw]

Inference optimizations I’d study if I wanted sub-second LLM responses:

1.KV-Caching
2.Speculative Decoding
3.FlashAttention
4.PagedAttention
5.Batch Inference
6.Early Exit Decoding
7.Parallel Decoding
8.Mixed Precision Inference
9.Quantized Kernels
10.Tensor Parallelism
11.Pipeline Parallelism
12.Sequence Parallelism
13.Graph Optimization (ONNX, TensorRT)
14.Dynamic Batching
15.Memory Offloading
16.Streaming Generation

[wh @nrehiew_]

For the visual learners

[fin @fi56622380]

华为τ scaling定律营销策略，无非是more than moore的广义摩尔定律的另一种说法而已

作为芯片架构师，我更感兴趣的，还是芯片密度提升，ppt上41%能耗提升和12.7%性能提升，到底是怎么实现的

看完了论文，感觉华为这次创新，本质上是用设计复杂度高 + 高制造成本 + 超前散热，一定程度弥补了工艺差距

-----------------

1. 华为芯片堆叠带来的等效密度提升，是虚假宣传还是真的，是不是工艺突破？有没有实打实的好处？

等效密度提升的来源，是两片芯片用hybrid bonding技术绑在一起，投影面积理论上能减小一半，但第一代不是全芯片双层折叠，而是选择性折叠关键logic，所以只有大概53%的芯片面积实现了折叠(密度155->238)，等到后面几代折叠面积会逐渐增大，到2030年接近全折叠（密度155->292）

这2026第一代等效密度从 2025 年 155 MTr/mm² 跳到 2026 年 238 MTr/mm²，时钟频率也提升了12.7%，功耗比提升41%，表面上看似乎和工艺突破没有什么区别，但有一点重要区别就是leakage power华为从头到尾没有提，只要工艺节点不变，gate leakage、junction leakage 不会因为 3D stacking 自动改善

2030年到2031年的等效密度突变，大概率是来自于2层堆叠到3层堆叠，正如2025到2026年的等效密度突变，时钟频率突变，来自单层到2层折叠

所以从leakage没提这个事来看，这个2031年等效1.4nm，和工艺节点上的突破没有联系。

本质上是用设计复杂度高 + 高成本 + 超前散热 + 超前部署advanced packaging，一定程度弥补了工艺差距

-----------
那么这样看起来虚假的等效密度提升，有用处吗？好处在哪里？

有的，设计上topology折叠，原来要跑几毫米的水平走线，折叠后变成了几十微米。降低了super buffer/bus的长度，降低了clock tree的深度（clock depth -42%、clock wire -28%），clock skew也带来了改良(-25%)，这对动态功耗的改善是实实在在的。部分critical path的缩短，也让时钟频率的上升更容易

所以ppt roadmap上performance的提升，从2025年到2026年上升了12.7%，大部分都是来自于时钟频率的上升（12.7%）

所以好处基本上是topology拆分电路逻辑设计上带来的提升

既然没有实质上的工艺提升，华为芯片堆叠带来等效密度提升的trade off代价在哪里？

三个代价：散热超前发展，设计复杂度高，制造成本变高

最大的代价就是热密度的同步上升，理论上logic on logic都是CPU execution发热最严重的区域，这部分折叠起来相当于功耗密度直接翻倍，但算上41% power efficiency改善，功耗密度仍只比非堆叠方案高40%左右。所以第一代只能对最关键的部分做折叠，大概只占全芯片面积的53%。

所以散热技术也被逼的超前发展，直接上毫米级的MEMS风扇，做micro-cooling fan。

另外的代价就是设计复杂度的变高，critical path的折叠，哪个部分的logic能折叠，折叠之后又会带来从前端到后端的巨大变化要推翻重来

现有的所有EDA工具也不可能支持3D topology，论文自己也承认，full-scale LogicFolding需要全新的3D-native EDA toolchain，把多层stacked dies当作单一连续设计实体处理。哪些logic能折叠、折叠后的inter-die timing closure怎么做，Physical Design（PD）也是难点

制造成本也会更高，被迫超前部署advanced packaging封装，1.5~2um的hybrid bonding + logic on logic都是很有挑战需要显著更高的成本
以前一层wafer做一次光刻；现在两层wafer分别做光刻再bonding，加上hybrid bonding的overlay控制（论文要求<0.5μm）、TSV、KOZ keep-out zone、冗余修复、良率乘法损失，每颗芯片的制造成本和测试成本都要显著上升

--------------------------

2. Tau scaling这个说法，scaling的到底是什么，这个scaling技术路线是不是一次性的design topology红利？潜力如何？持续进步的空间在哪里？

τ Scaling的核心主张是：用时间常数τ替代几何线宽作为全栈优化目标，在器件、电路、芯片、系统四个层级分别压缩特征延迟

公式本身没有任何新物理。"关注瓶颈延迟"是所有架构师都在做的事情。整个行业都知道互联RC是延迟瓶颈，TSMC每一代工艺都在用low-k dielectrics/semi-damascene等手段降RC。把一个众所周知的优化方向包装成"定律"是显然的营销宣传手段，本质是More than Moore的广义摩尔定律的另一种说法

抛开marketing，华为目前所谓RC delay的改善，本质上是芯片堆叠之后，topology距离缩短，让匹配的effective RC都变小，不是RC工艺常数

至于scaling的意思，是能持续发展的一条roadmap。这里的持续改善路径指的是，全芯片堆叠的层数越来越多，从25~30年的2层堆叠，到31年开始的3层堆叠，以后甚至会考虑4层堆叠

第一代折叠技术甚至不是全芯片双层折叠，而是选择性折叠关键logic，所以只有大概53%的芯片面积实现了折叠(密度155->238)，等到后面几代折叠面积会逐渐增大，到2030年接近全折叠（密度155->292）。2031年的roadmap之所以会出现一个阶跃，就是因为那是从2层折叠到3层折叠的时间点。

但需要注意的是，这个scaling方法的边际效应是逐渐缩小的，折叠成双层的收益是100%，2->3层的收益就只有50%，如果2035年再从3->4层堆叠，收益就只有33%了

另外随着堆叠层数变高，上面说到的三个挑战，散热，设计复杂度，成本，都是越来越大

---------------------

3. 华为的芯片堆叠，是不是TSMC/AMD已经有的hybrid bonding技术？华为做到的是cache on logic，cache on cache，还是logic on logic，logic on logic最大的散热问题是怎么解决的？

是已经有的技术没错，但同时也是把现有技术指标做到了领先也是真的，3D堆叠本身不是新技术，TSMC的hybrid bonding量产还是6um，华为论文给出Kirin 2026的hybrid bonding pitch是1.5μm

我在刚刚看到华为的堆叠消息之后，第一反应也是怀疑和AMD的3D V cache类似，它主要把 SRAM cache 叠在 已经有的L3 cache 区域上，通常会避免直接堆在最热的 CPU execution logic 上，就是避免散热问题，毕竟SRAM 的功耗密度和热点特性与high-activity logic 不一样，如果最热的logic on logic堆叠，散热恐怕会碰到困难

但看了更多数据之后，clock buffer -56%、clock depth -42%、clock wire -28%，这些只有在core内部的clock distribution被重构时才可能发生。纯SRAM stacking不会碰core内部的clock tree。另外如果只是cache on cache，大概率是不需要单独MEMS微型风扇额外散热的，证据普遍都指向logic on logic方式

华为这个技术的精妙之处在于，logic on logic 折叠之后热密度并没有翻倍，而是因为topology的好处，能耗下降了30%，这样热密度只上升了40~50%

而第一代没有完全把整个最热的execution logic 100%堆叠起来，论文也明确说selectively applied along key critical paths，只是大概53%有选择性关键路径会堆叠起来，可能颗粒度都没有那么好，只是IP堆叠在IP上，那么热密度上升也许能维持在20%以内

但这条道路继续前行，超前发展的散热就成了必然，现在是MEMS微型毫米级的主动散热风扇，紧贴处理器传导效率高，和华为手机一样，散热堆料特别足，而且技术领先同行。

以后怕是要把HBM7/8的微流道散热技术提前用起来了，毕竟HBM7/8要上24+层堆叠，华为很可能要在提前用上下个世代的散热技术了

-------------------------

4. 从架构角度来说，最重要的问题，华为41%的power efficiency（能耗比）提升，到底是怎么实现的？为什么AMD的3D V cache没有这么大的提升？

首先确定41%的定义。论文只说"SoC performance-core power efficiency improved by 41%"，没有给出benchmark名称、Voltage/Freq点、温度条件、功耗边界。但PPT roadmap上有一个关键线索：ISO-Power Performance的数字，2025年是2.75，2026年是3.1，提升12.7%

这个时钟频率提升12.7%完全一致，可以理解为，同功耗的性能提升是12.7%，绝大部分是时钟频率提升带来的

至于能耗比上优化的猜测是，LogicFolding缩短critical path → 在固定Vdd下Fmax从2.75GHz提升到3.1GHz → 这意味着在原来的2.75GHz频率下，有了约12.7%的timing headroom → 这个空间在iso-performance模式下可以换成更低的Vdd

另外的能耗比的提升，可能也来自于电路折叠之后，cache hit latency的下降。从业界经验来看，一般L2/L3 cache hit latency下降10%，CPU整体性能会有至少5%的提升

ppt里显示SRAM latency下降30%，估计会有一部分转化为cache hit latency的下降

AMD的3D V cache没有这么大的提升，主要是因为AMD的底层logic die并没有重新设计，3D cache的延迟latency不仅没有减小反而加大，只是增加了cache大小，收益不如latency下降那么明显。

另一方面，clock skew的下降,critical路径变短，造成电路timing变好，意味着华为可以使用更低的vdd（猜测甚至能低7~8%），以及路径缩短所带来的RC的下降(考虑到clock buffer -56%、wire -28%、SRAM pJ/bit -24%这些数字，比如C_eff下降10~15%合理)，再加上clock tree的整体缩短和下降，确实是有可能在部分Voltage/Freq点做到同性能下，做到30%的功耗下降的，而30%的功耗下降换算过来就是41%的power efficiency

对比苹果和高通，每一代手机芯片在iso-power下单核性能一般提升10-20%，iso-performance下功耗一般降30-40%，这是V/F曲线的特性决定的，所以从经验上来说，数字是对的上的。

所以这个power efficiency（能耗比）的提升，从现有的数字上来说可以从topology推导出来是合理的，可能真的和工艺节点没有太大关系

----------------------------

5. 这个技术路线有没有可复制性，其他家会不会效仿？

短期内不会大规模效仿，因为性价比和风险收益比来说不好。长期来看，这个方向所有人都在走，只是名字不一样

华为做LogicFolding的根本驱动力是制裁，工艺节点被卡在7nm，只能在封装，散热，和设计层面想办法弥补。华为也为此付出了不小的代价：散热成本，设计复杂度，以及制造成本更高（包括良率）。这是一个被逼出来的路线，不是一个自然选择

其他玩家在用TSMC就能做到正常的经济迭代，是没有必要冒着这个风险，去超前迭代散热技术和设计复杂度的

长期来看，Intel的Foveros、TSMC的SoIC、AMD的MI300的3D stacking都在朝同一个方向走。如果继续追最先进节点的经济性持续恶化，那么"固定一个成熟节点+3D topology optimization"的路线会越来越有吸引力

散热方面，MEMS微型风扇和微流道也会成为未来HBM散热的主流

-------------------

总结一下，华为这次的创新，绝对是值得尊重的，在制裁环境下，用极高的设计复杂度和成本，在一个被锁定的工艺节点上大胆重新设计，榨出了一次大的topology红利，虽然它有天花板。每多加一层的边际收益递减（堆叠1->2层, 2->3层, 3->4层，提升百分比变小），leakage无法解决，散热越来越难，3D EDA工具链更是全新的挑战。

但这个Tau scaling不是一条可以走十年的指数增长路径，每次爬完一个台阶，下一个台阶更难爬，而且台阶更矮收益更小，华为以后想缩小差距，还得再想想靠什么其他的路线

[Roan @RohOnChain]

https://t.co/ylkXptDWTw

[Paul @paulWilliamChan]

I wrote a matmul kernel on B200 in pure CUDA/PTX that beats cuBLAS by 6% at M=N=K=8192.

Inspired by @gaunernst's blog on Blackwell instructions with benchmarking done on @modal.

Blog: https://t.co/nifzaMdPJS
Repo: https://t.co/EKjrnCrp0z

[Russ Salakhutdinov @rsalakhu]

New work on Scaling Test-Time Compute for Agentic Coding:

Paper: https://t.co/1bQXq7bwDk

This work introduces a test-time scaling framework for agentic coding that converts rollouts into structured summaries capturing key hypotheses, progress, and failure modes while discarding low-signal details.

This enables two forms of inference-time scaling: (1) Recursive Tournament Voting (RTV) for parallel selection via iterative small-group comparisons, and (2) Parallel-Distill-Refine (PDR) for sequential improvement by conditioning new rollouts on distilled summaries.

Our approach consistently boosts performance on frontier benchmarks. On SWE-Bench Verified, Claude-4.5-Opus improves from 70.9% → 77.6%, and on Terminal-Bench v2.0 from 46.9% → 59.1%. These gains highlight that effective test-time scaling for long-horizon agents hinges on representation, selection, and reuse, not just sampling more trajectories.

Check out a more detailed thread by @anirudhg9119.

[Akshay 🚀 @akshay_pachaar]

CPU vs GPU vs TPU vs NPU vs LPU, explained visually:

5 hardware architectures power AI today.

Each one makes a fundamentally different tradeoff between flexibility, parallelism, and memory access.

> CPU

It is built for general-purpose computing. A few powerful cores handle complex logic, branching, and system-level tasks.

It has deep cache hierarchies and off-chip main memory (DRAM). It's great for operating systems, databases, and decision-heavy code, but not that great for repetitive math like matrix multiplications.

> GPU

Instead of a few powerful cores, GPUs spread work across thousands of smaller cores that all execute the same instruction on different data.

This is why GPUs dominate AI training. The parallelism maps directly to the kind of math neural networks need.

> TPU

They go one step further with specialization.

The core compute unit is a grid of multiply-accumulate (MAC) units where data flows through in a wave pattern.

Weights enter from one side, activations from the other, and partial results propagate without going back to memory each time.

The entire execution is compiler-controlled, not hardware-scheduled. Google designed TPUs specifically for neural network workloads.

> NPU

This is an edge-optimized variant.

The architecture is built around a Neural Compute Engine packed with MAC arrays and on-chip SRAM, but instead of high-bandwidth memory (HBM), NPUs use low-power system memory.

The design goal is to run inference at single-digit watt power budgets, like smartphones, wearables, and IoT devices.

Apple Neural Engine and Intel's NPU follow this pattern.

> LPU (Language Processing Unit)

This is the newest entrant, by Groq.

The architecture removes off-chip memory from the critical path entirely. All weight storage lives in on-chip SRAM.

Execution is fully deterministic and compiler-scheduled, which means zero cache misses and zero runtime scheduling overhead.

The tradeoff is that it provides limited memory per chip, which means you need hundreds of chips linked together to serve a single large model. But the latency advantage is real.

AI compute has evolved from general-purpose flexibility (CPU) to extreme specialization (LPU). Each step trades some level of generality for efficiency.

The visual below maps the internal architecture of all five side by side.

👉 Over to you: Which of these 5 have you actually worked with or deployed on?

[Jared Roesch @roeschinc]

Many of us are heavily using Codex every day for weeks/months to build real features across frameworks, systems, compilers, dev-tools, and more. It has been crazy to see millions of tokens rapidly turn into features. The last few releases have made systems work finally viable.

[Chris Lattner @clattner_llvm]

Cool to see that Tesla Full Self Driving has adopted the @LLVMFoundation MLIR stack, and is seeing 20% faster reaction time as a result. It is quite likely that a modern compiler and runtime implementation the break-through that robotaxi and FSD have been waiting for!

[Diyi Yang @Diyi_Yang]

Cool to see Chronicle from @OpenAI 👀This is closely related to what we explored in GUM. Great to see these ideas making it into products!!! https://t.co/r9eTsyCU3e

[Garry Tan @garrytan]

YC will invest in any YC company in stablecoins

The new financial rails of the revolution will not be over ACH or wire

[墓碑科技 @mubeitech]

英伟达换一次半导体工艺，迁移三千个标准单元库曾经需要8个人干10个月。
现在单张GPU跑一夜就能搞定。
最终产出的功耗和延迟数据，直接盖过人类工程师的图纸。

这家公司把过去三十年历代GPU的底层源码和绝密文档全盘喂给AI。
练出了一个专供内部的超级大脑ChipNeMo。
新手遇到不懂的底层原理直接找大模型连环追问。
资深老手彻底从基础答疑里解放出来。
连抓Bug、分析错误源头、给具体模块指派责任人的活，全由AI一手包办。

进位先行链是一个从上世纪五十年代就被反复研究的底层难题。
英伟达让强化学习程序像打街机游戏一样疯狂试错。
它只要勉强满足时序底线，就把体积和功耗往死里压。
最终系统吐出的架构图纸诡异至极。
人类设计师根本画不出那种走线。
测试台上的硬性指标硬生生比人工精调的方案高出30%。

如今英伟达内部的AI群体已经在接管新架构的参数探索。
首席科学家Bill Dally描绘了硬件制造的终局。
开局甩给系统一句指令：去把下一代GPU设计出来。
人类工程师直接去度假滑雪。
几天后回来，拿着跑完验证流程的完美图纸找台积电流片。

[Pranjal Aggarwal ✈️ ICLR'26 @PranjalAggarw16]

What if computer-use agents could do real work?

We built Gym-Anything: a framework that turns any software into a computer-use agent environment.

We used it to create CUA-World: 200+ real software, 10,000+ tasks and environments, across all major occupation groups, from medical imaging to financial trading.

🧵

[WquGuru @wquguru]

这两本书把Claude Code和Codex的Harness工程吃透了，对于Claude Code和Codex源码解析和对比都入木三分，文科生也能读的津津有味：

《Harness Engineering——Claude Code 设计指南》：不是源码注释汇编，也不是产品功能介绍。它关注的是 Claude Code 如何把不稳定模 型收束进可持续运行的工程秩序，让控制面、主循环、工具权限、上下文治理、恢复路 径、多代理验证与团队制度长成一套完整骨架

《Claude Code 和 Codex 的 Harness 设计哲学——殊途同归，还是各表一枝》：比较两套 AI coding harness，最容易犯的错误，是拿一张功能对照表当作思想史。左边写“有技能”，右边也写“有技能”；左边写“有沙箱”，右边也写“有沙箱”；左边写“能开子代理”，右边也写“能开子代理”。这样写的好处是省事，坏处是几乎什么也没说。因为工具中的名词相同，不代表系统的骨架相同。就像两个城市都修了桥，不能说明它们是按同一条河设计的。

Github仓库：https://t.co/qVWdzLJP72
在线阅读：https://t.co/PoyAGfZHDA

[YC (Yucheng) @yucheng]

Facebook 首批工程师、Dropbox CTO，用 AI 编程一个周末之后说：我们再也不会手写代码了。

接下来 5 天，他写的代码比过去 5 年加起来还多。

他在 South Park Commons 见过几百个工程师和创始人，发现工作年限和 AI 适应能力几乎没关系。FAANG + 名校背景，几乎什么都预测不了。面试里故意给一个手动完不成的任务，用 AI 的人和没用的，代码量差距是 10 倍。

最有用的筛选信号：有没有个人网站，有没有 side project，有没有那种看到新工具必须今天就用上的性格。他管这叫 restlessness（坐不住）。他现在投人就押这个。