Yiwen Hu

PhD at GSAI, RUC

[Niels Rogge @NielsRogge]

12 million views for a JSON classifier?

Yeah, we're in a bubble

[Yiwen Hu @hungry2e]

@xidulu @JohnCLangford Why multiply e_t by W^G? Is there an intuitive explanation?

[Yiwen Hu @hungry2e]

@xidulu @JohnCLangford Is there a better approach than sampling different numbers of passes? For instance, recurrently training on a fraction of data in a decoding-like manner.

The former still leaves a train-inference gap, while the latter keeps all inference scenarios in-domain.

[Xidulu @xidulu]

1/ Sharing a new, interesting project I did during my internship at Microsoft AI Frontiers w/ @JohnCLangford

TL;DR: At decoding time, we feed **previous hidden state** into the input together with token embedding, and it boosts performance for free.

https://t.co/za3cZ4UwWV

[Yiwen Hu @hungry2e]

@totheagi I think the real bottleneck with the 5090 is that PCIE speed tops out at 32 GB/s

[AI Box @RUC_AIBox]

SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training

SWE-Master is an open-source and reproducible post-training framework for building strong software engineering agents, covering data synthesis, SFT, RL, and TTS.

📑 https://t.co/c4aOej5gZl

[AI Box @RUC_AIBox]

New preprint: "ClawGym: A Scalable Framework for Building Effective Claw Agents"

TL;DR: ClawGym is the first unified framework for large-scale data construction, capability training, and reliable evaluation of Claw Agents.

📑https://t.co/KjeUFGX3vS
💻https://t.co/osgirkifw5

[AI Box @RUC_AIBox]

Introducing "Towards Long-horizon Agentic Multimodal Search"

TL;DR: LMM-Searcher is a multimodal search framework that leverages a file system to manage visual context, enabling effective scaling to 100-turn interactions.

📑https://t.co/bR2pdl5Fip
💻https://t.co/RQXVmsg1B9

[You Jiacheng @YouJiacheng]

Inspired by @Jianlin_S , I found a better proof for the convergence (to SVD) of the iteration in this blog.
It turns out that ALL intermediate iterations are just preconditioners for the final QR.

[Jianyang Gao @gaoj0017]

The TurboQuant paper (ICLR 2026) contains serious issues in how it describes RaBitQ, including incorrect technical claims and misleading theory/experiment comparisons.

We flagged these issues to the authors before submission. They acknowledged them, but chose not to fix them. The paper was later accepted and widely promoted by Google, reaching tens of millions of views.

We’re speaking up now because once a misleading narrative spreads, it becomes much harder to correct. We’ve written a public comment on openreview (https://t.co/nDVjmNhATM).

We would greatly appreciate your attention and help in sharing it.

[fin @fi56622380]

复盘GTC 2026：Nvidia补上了短板，大幅削弱了各个AI 加速器 startup最大的优势--token速度

回顾这篇GTC前瞻，方向预测和技术路线写的没啥大问题，最后Nvidia给出的解法比我想象的更为精巧：不仅是prefill放在GPU上，decode阶段Attention阶段也放在GPU上(这点没想到)，只把decode的MLP阶段放在LPU上做

这和MatX的解决方法有异曲同工之妙，Weights放在SRAM上，KV cache放在HBM上

这样的好处在于，Attention阶段需要巨量的KV cache(动辄几十上百GB)，本就是LPU SRAM无法承受的，把这部分放在HBM上是更合理的选择

正应对了未来agentic flow里多轮对话上下文长，long context KV cache爆炸的趋势，即便是高batch并发数产生巨量KV cache也能让HBM容纳。随着上下文长度变长，所有的增量成本都在GPU HBM上，LPX是完全静态的不受影响，只和模型本身大小相关

让LPU宝贵的128GB SRAM只承担FFN/MLP阶段的固定weights，而FFN阶段占GPU整个decode阶段的50%以上，如果是短context甚至能占比超过60%，FFN这部分在LPU上得到数倍大幅加速，是很划算的

这样设计的部分drawback可能在于，一般transformer的decode阶段有很多层，比如以80层为例，那就是attention层和FFN要重复80次，也就是说，tensor要在GPU和LPU之间互相传递80次，虽然中间是low latency Nvidia Spectrum-X Ethenet，但生成一个token需要80次GPU-LPU往返延迟累加，这也是不小的损耗

这样的新架构，按attention和FFN各占40%/60%来算，FFN阶段加速几倍，极限最高速度来说，整体加速能达到一倍以上（和Rubin NVL 72比）

最高速度突破1000 token/s的同时，还能让整体throuhput仍然能保持一定的商业价值。要知道如果在Blackwell要跑到400~500 token/s高速，只能同时处理很少的几个请求，这对GPU资源是巨大的浪费。而现在就算是跑到1000 token/s，也能保持一定的batch size(吞吐量)了，终于也能产生商业价值了

图里说在400 token/s的速度下，Rubin + LPX把吞吐提升了35倍，就是典型的token经济学，这个token高速度下，从Blackwell算提升了35倍的商业价值

-------

Nvidia补上了这块短板之后，对各个startup（比如Cerebras, d-Matrix, MatX, SambaNova）有什么影响呢？

startup最大的卖点就是特定场景下的速度优势，或者成本优势

在大batch（多请求）场景下，GPU的算术密度(arithmetic intensity)越过ridge point之后利用率接近很高，成本/速度都对startup有显著的优势。

所以这些startup能存活，最大的场景是：客户的workload集中在小batch、低延迟，速度极快，不在乎极高成本。GPU在这里效率极差，也达不到对应的token速度

Cerebras：极致的速度。wafer-scale尺寸巨大的SRAM(40GB)，消灭芯片间通信这个最大瓶颈，在小batch用户数量小的场景下token rate极高。但成本完全没有竞争力，一台CS-3系统价格230万美元，远超同等GPU集群，跟H100比是十几倍的成本换十几倍的速度。

d-Matrix ：高速度+小batch场景。in-memory compute减少data movement，在小batch decode下比GPU的利用率高，所以perf/watt在这个区间有一定竞争力。最近引入的3D stacked DRAM 是为了解决“更大的 reasoning model + 更高 token consumption”带来的容量/带宽继续扩展问题

SambaNova ： 在企业私有化部署场景下，同时跑多个中小模型，GPU的利用率因为context switching损耗严重，SambaNova的RDU在这个场景下有更好的perf/dollar。本质上还是特定场景下的成本优势，通用速度优势并没有那么大

MatX：partitionable脉动阵列 + SRAM/HBM混合，和这次Nvidia的异构架构思路有相似的地方，最大的亮点是单芯片内实现Weights放在SRAM上，KV cache放在HBM上。但单芯片内省掉了前面提到的AFD的80层LPU-GPU芯片间通信，所以速度上仍然有一定优势，但Scalability可能不如GPU+LPU阵列了

总之，在Rubin + LPX情境下，小batch、低延迟，速度极快这个以前的场景缺口补上了很多，各个startup的优势空间越来越缩小了

-------------

前瞻里提到的speculative decoding用LPU做草稿模型，用GPU去验证，这样的加速幅度会非常大，这个猜想完全命中，这次在官方blog里有了浓墨重彩的一笔，专门用了一整个章节来介绍这个用法：“LPX generates draft tokens rapidly using its low-latency architecture. Rubin GPUs verify and finalize tokens efficiently”

另外一个前瞻里提到的CPX (Content Phase aXcelerator，一个专门为prefill的compute bound特性设计的计算模块)，似乎在这次GTC里完全消失了，一个字也没提，这是意味着CPX被彻底取消了吗？

我觉得不一定

目前的prefill和decode是disaggregated结构，也就是说一部分的GPU专门做prefill，另一部分专门做decode。CPX取代GPU做prefill从架构上来说是更合理的选择，可以加速prefill阶段，当然了会带来更高的成本，毕竟也是额外的一颗芯片

CPX和目前Nvidia的Rubin + LPX架构没有冲突的地方，仅仅只是简单的把做prefill的这部分GPU换成CPX而已，所以以后有速度优化需求的时候，也许CPX还会回来的

—-------------------------------------------------------

还是上篇的感慨，每一次计算范式的改变，半导体都会带来一波新的startup热潮，但当软件/应用形态逐渐收敛，最后还是变成了大厂通过收购把功能做大做全，参数做的更高，系统深度整合的更好更全面，成本更低，功耗和跑分更优秀，让startup慢慢失去独立生存的空间

比如移动互联网时代早期，也是群雄并起，有做AP应用处理器，独立基带芯片的，ISP的，GPU的各种小公司。但最后的赢家，都是从到后来把GPU，ISP，modem全都做进SoC，并且完成系统级整合的异构计算平台。

苹果收购PA semi的CPU，英飞凌的modem，掏空Imagination的GPU；高通收购ATI的mGPU，Atheros的Wifi，Nuvia的CPU，CSR的蓝牙/DSP，都是典型例子

异构推理的复杂度越来越高，能做系统级整合的公司会更有优势，这和移动SoC时代的逻辑一模一样。AI时代nvidia收购arm(失败)，收购Mellanox，收购groq，只是这个新历史轮回的开始

[Fengzhuo Zhang @FengzhuoZhang]

Large Language Models (LLMs) exhibit “slash patterns” in attention maps — a key mechanism behind prefilling acceleration.

We take a first step toward understanding why they emerge.

Main findings:
▶️ Slash patterns are OOD-generalizable
▶️ Queries and keys on these heads are near rank-one and carry little contextual information.
▶️RoPE is the primary source of the slash pattern.

Blog link:
https://t.co/uhE3y7i5xW

A thread 🧵

[Tian Xie (Unakar) @unakar666]

A fundamental solution to LLM training instability: Spectral Sphere Optimizer (SSO)

Design an optimizer fully from μP principles.

Outperforms AdamW and Muon on Dense 1.7B, MoE 8B, and 200-layer DeepNet

Paper: https://t.co/ODVW4FC3R5

Code: https://t.co/zFaXi2YpkS

[Yiwen Hu @hungry2e]

5/
Incentives matter too.

Why should people share compute, bandwidth, or sensor data with robots?

Web3 offers tokens, DAOs, and marketplaces—basically, ways to align strangers around a shared machine economy.

[Yiwen Hu @hungry2e]

4/
Then comes coordination.

Swarm robots, supply chains, logistics.

Smart contracts could be the neutral “rules of the game,” avoiding dependence on one company’s servers.

[BangLiu @BangL93]

🧠264 pages and 1416 references chart the future of Foundation Agents.

Our latest survey dives deep into agents—covering brain-inspired cognition, self-evolution, multi-agents, and AI safety.

Discover the #1 Paper of the Day on Hugging Face👇:

https://t.co/CvtqGKyCYb

1/3

[Yiwen Hu @hungry2e]

@Yong18850571 just curious, is the RL version using multi-turn conversation?