Alex Zhang

AI Infra Engineer

[Alex Zhang @AlexZhang152005]

We built this: formal verification for LLM serving, driven by coding agents. It found real concurrency bugs and give performance optimizations in production LLM serving frameworks. Give it a try!

[Hao AI Lab @haoailab]

Towards Verifiable LLM Infra 🚀

Suppose coding agents can build a fast inference engine from scratch, and one engineer orchestrates 10,000 agents in parallel: how do we know the engine they wrote is actually correct? Having agents review other agents mostly wastes tokens compared to empirical and formal verification.

We introduce vstack, a 5-stage formal verification workflow for coding agents, and veriserve, a verifiable PD-disaggregated serving engine: uncovering 3 classes of real RDMA and cache races in production serving engines, and backing veriserve's KV-page lifecycle design model with a 226-module, 516,973-obligation TLAPS safety proof at +0.18% mean p50 TPOT overhead on GB200.

Blog: https://t.co/SdbINvSVht
veriserve: https://t.co/ugHLToReCN
vstack: https://t.co/GZJebbjdpL

[Alex Zhang @AlexZhang152005]

Agentic formal verification found lots of bugs that hide in rare feature combinations
Is this the future of LLM infra, or even ALL software? 👇

[Hao AI Lab @haoailab]

(1/7) Agentic Formal Verification on vLLM 🚀

This week, we introduced vstack for agentic formal verification in AI infra; today, we're working with @vllm_project , putting vstack to test: although vLLM is already very mature, some bugs only surface in rare combinations that are hard for human reviewers to catch.

With vstack's agentic formal verification, we identified 17 (and growing!) bugs, from silent wrong output to server lockout. Below we share 5 of them, each with a fix PR ⬇️

🔌 vstack plugin: https://t.co/GZJebbjdpL
📋 Bugs Tracker: https://t.co/6f6WmtRUWU

[Tianyin Xu @tianyin_xu]

Cool work from @haoailab that uses TLAPS to verify TLA+ models extracted from source code of LLM serving engines. This's probably the single largest use case of TLAPS to date. It shows that TLA and state-machine reasoning remains extremely useful for AI infrastructure. Concurrency is just a hard problem.

[Google DeepMind @GoogleDeepMind]

For the first time, AI has mathematically proven @signalapp’s protocol is secure down to the code level.

Using our breakthrough AlphaProof Code framework, we verified the messaging app’s cryptographic system – which protects billions of private messages every day.

[Alex Zhang @AlexZhang152005]

@haoailab 🚀🚀🚀

[shawn. @zzzzshawn]

Agent diff chart

[Vincent Zhong @vincentzed_cuda]

A chip that’s 3× faster but idle half the day isn’t 3× cheaper:

A prefill or decode device only pays off when your traffic matches the ratio you bought, and that ratio keeps changing. Your GPUs can switch roles when that happens, but specialized silicon just sits idle while you keep paying for it. Software keeps moving the bar too, since megakernels alone have pushed low-concurrency decode on ordinary GPUs up by 40–75% this year, and every one of those gains is another knock on specialized chips.

Consider: “how fast?” but “how fast, times how often it’s actually busy, after moving the KV cache over the network?” If your GPUs run at 70% and their chip averages 40% across a real year of traffic, it needs to be ~1.75× better just to tie. For a single model with very stable traffic (like a premium fast tier), that can work out, but in many case, general-purpose computing can be more useful.

[Alex Zhang @AlexZhang152005]

@alokbishoyi97 interested

[Simran Arora @simran_s_arora]

Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨

Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.

Together Compiler research highlights include:
1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems.
2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2.
3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation.
4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.

From an incredible team @reyna_abhyankar @dylan__lim @realDanFu @togethercompute!!

[Arnold Xu @realarnoldxu]

we open sourced the @flappyairplanes megakernel.

they raised $1B on the back of a demo video: one GPU kernel that trains a model, serves it, and plays Doom at the same time

me and @Bobliuuu reproduced it from just the video frames and open sourced it; see this thread for details 📷

code: https://t.co/8e6R6PwVal
video below

[Xingyu Dang @xingyudang]

Same recipe

[Kevin Z @kzdagoof]

How good are frontier models at writing Polytopia bots?

Introducing Polytopia Bench.

12 models each get 40 minutes to write a Rust bot from scratch.

[LiLens @LiLensss]

🚨 真正的赛博套娃！有人在千问权重的肚子里，硬生生编译了一颗英伟达 GPU。

开发者 @TheVixhal 开源了极其逆天的项目 qwenGPU：它不是外挂插件，也不是虚拟机，而是把一张完整的 SIMT 架构 GPU 编译成了 45 层神经网络权重，直接插在 Qwen3.5-2B 面前！

仔细看这个变态级的硬核架构：

1️⃣ 纯数学编译，把网络层当寄存器用：
这 45 层不是训练出来的，全是编译器算出来的。它把千问的 Gated DeltaNet 状态直接当成硬件寄存器和全局内存，自带 4 通道 Warp、4 级流水线、每通道 15 个寄存器和 126 字显存。

2️⃣ 模型前向推理，真在跑底层 CUDA：
把标准 CUDA 代码用 Clang 编译成 NVIDIA PTX 汇编喂给模型，千问在每一步 Forward 时逐 Token 模拟硬件时钟周期。实测 49 个 CUDA 算子与真实英伟达 Tesla T4 跑出来的结果逐字 100% 吻合！连分支、循环和 __syncthreads() 同步栅栏全支持。

3️⃣ 评论区直呼赛博套娃：
有人问：“那岂不是能在模型肚子里跑另一个模型？”作者坦白回答：“如果模型能压到 126 个字，跑个 2-2-1 神经网络毫无压力，套娃完全成立！”

为了解决大模型算术不行的毛病，直接在它脑子里焊了一颗 GPU。极客玩起底层，脑洞真的深不见底。

📎 演示视频与情报来自 @TheVixhal ，仓库：https://t.co/1TupHuMjbG

[Shiyi Cao @shiyi_c98]

1/n
Introducing our NeurIPS’26 work 💙Evolving Agent Teams (EAT) 💛, where a supervisor diagnoses why an agent team stalls and restructures it to escape local optima, for long-running optimization tasks.

EAT shows superior performance across domains:
🏆 Open Kernel Challenge, decode layer on H200: 2.449× over SGLang, #4 of 1,180 (as of Nov 1, 2026)
⚡️ NVIDIA kernels: KDA prefill on B200, 3.44× over FlashKDA (previous best: 2.96×)
🔥 AMD kernels: KDA prefill on MI350X, 3.01× over vLLM
💻 MoE serving on Mac: Qwen3.8-Flash-Next (Unsloth UD-IQ4_XS GGUF) on M5 Pro, ~3.3× faster prefill (930 tok/s) and 1.22× faster decode (43.7 tok/s) than Strata at the same memory
🦾 RoboDojo: 62.1 on held-out seeds (the best leaderboard entry as of Oct 26, 2026 is 55.3)

- Paper: https://t.co/KeSz8RyTne
- Code: https://t.co/cdLATp2jpw
🧵👇

[James Whittington @jcrwhittington]

Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.

Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.

With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington

[サイバー環状線 @CyberKanjousen]

《4 年前的古人第一次接触 ChatGPT 连接 agent 的珍贵聊天记录》

[外汇交易员 @fxtrader]

经典回顾：乔布斯解释为何触屏Macbook行不通。
#MacBookPro

[Ryan Peters @ryanirl]

A small swarm versus a single agent in an autoresearch harness

[Albert Gu @_albertgu]

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.

Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?

Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

[lifcc @mylifcc]

如果留给人类按传统方式做智力贡献的时间只剩 1–5 年，你现在手头的事还值得做吗？

扩散模型（Diffusion Models）的发明者Jascha Sohl-Dickstein在斯坦福大学做的一场半开玩笑、半认真的「劝世演讲」，针对的是年轻科研人员和技术从业者。

他指出了一个普遍的精神分裂（认知不一致）：
• 很多人脑子里觉得 「AGI 未来 1–5 年就会到来，世界将天翻地覆」；
• 但行动上却还在按 「花 5 年读博 → 熬博后 → 评终身教职」 或者 「花一两年慢工出细活憋一个传统项目」 的旧时代节奏规划人生。
他的核心忠告是：如果你真信 AI 进化是指数级的，现在就必须改动作：
• 别做慢周期、顺风车式的修补项目：如果一个问题靠「等下一代模型发布」就能自动解决，现在花一年去做它毫无意义（即 Sutton 的《苦涩的教训》）。
• 项目周期要短于模型代际更新：快速交付，做能随着模型变强而水涨船高（借力规模）的事。
• 把自己训练成「导师（PI）」：每天强迫自己用模型，练习清晰拆解任务、审查结果，而不是自己闷头手搓所有细节。