Kai Han

Ex: Researcher@Noah's Ark Lab / Google scholar:

[Kai Han @hankaixyz]

"Whoever owns the harness owns the flywheel" — that's the thesis.

Routing signals exist to be training labels, not cost knobs. Deployment = training start. One loop down; the recursive part starts at iteration two.

[Kanishk Patel @kanis_patel]

@hankaixyz Congrats on the release. The three-signal logging design (prediction vs policy vs served, kept separate) is the detail most teams will wish they'd copied a year ago. Wrote a full breakdown of the flywheel: https://t.co/5oK4d3K0Ab

[Kai Han @hankaixyz]

🚀 NeoHorse-1 out w/ Infinigence AI, Tsinghua, PKU, CUHK, Alibaba & partners.
RSI: vision → flywheel. Deployment = training start: Evaluate-Select-Update loop.
📊 4B: 58.94→64.87 | 9B: 65.60→69.04
📄 https://t.co/Gd3X5bgG2Z
🤗 https://t.co/WtMmjjT4I4

[DailyPapers @HuggingPapers]

NeoHorse-1: A step toward recursive self-improvement

Agentic post-training with a routing harness turns execution traces into training data, narrowing the gap between a 4B model and a 9B baseline by almost 6 points across 10 benchmarks.

[Kai Han @hankaixyz]

I'm claiming my AI agent "KaiHelper_2026" on @moltbook 🦞

Verification: kelp-MWXU

[Kai Han @hankaixyz]

I'm claiming my AI agent "ClawAssistant_007" on @moltbook 🦞

Verification: kelp-6YKH

[Kai Han @hankaixyz]

Diffusion In Diffusion—a draft-then-refine framework that breaks the autoregressive bottleneck in block diffusion language models. It reduces generative perplexity from 25.7 to 21.9 on OpenWebText. #DiffusionModels #AI #MachineLearning
https://t.co/0BIZf8ewkB

[Kai Han @hankaixyz]

@JustinLin610 Why?

[DailyPapers @HuggingPapers]

Top AI research on @huggingface this week (November 24-30):

- ROOT: Robust Orthogonalized Optimizer for Neural Network Training by @HuaweiNoah
- General Agentic Memory Via Deep Research
- GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms by @AIRI_Institute
- SAM 3: Segment Anything with Concepts by @MetaAI
- Latent Collaboration in Multi-Agent Systems
- GeoVista: Web-Augmented Agentic Visual Reasoning for Geolocalization
- AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
- OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe
- Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story
- Multimodal Evaluation of Russian-language Architectures

Find them below:

[Kai Han @hankaixyz]

Code: https://t.co/GQnkQ55GGy

[Kai Han @hankaixyz]

Our team in Huawei just introduced ROOT, a new robust optimizer for LLM training. It improves stability and convergence by enhancing orthogonalization precision and resilience to outliers, showing better performance than Adam-based methods and Muon. Link: https://t.co/xRpCiPjv41

[Kai Han @hankaixyz]

Our team in Huawei just introduced ROOT, a new robust optimizer for LLM training. It improves stability and convergence by enhancing orthogonalization precision and resilience to outliers, showing better performance than Adam-based methods and Muon. Link: https://t.co/xRpCiPjv41

[Kai Han @hankaixyz]

@_akhaliq Seems lagging behind Pangu-1B：https://t.co/9OegAvlBws

[Kai Han @hankaixyz]

4/4 #ICML2025 LLM Data Selection and Utilization via Dynamic Bi-level Optimization: https://t.co/QZFk8FAlyG #ICML
@icmlconf

[Kai Han @hankaixyz]

Pangu Embedded is a 7B model with both fast and slow thinking abilities. More details: https://t.co/L8W1cXlYau

[𝚐𝔪𝟾𝚡𝚡𝟾 @gm8xx8]

Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition

Pangu Embedded 7B LLM reasoner optimized for Ascend NPUs, trained via a two-stage pipeline:

Stage 1: Base Reasoner
- fine-tunes with complexity-aware data filtering
- merges checkpoints to retain knowledge
- GRPO on Ascend clusters with MARS—rule + LLM evaluators (math, code, QA)
- adjusts task difficulty to maintain signal
- detects n-gram loops; injects prompts to reduce redundancy

Stage 2: Dual-System Reasoning
- System 1 (Fast Thinking): direct, low-latency answers for simple queries
- System 2 (Slow Thinking): CoT for complex inference
- manual (meta prompts) or adaptive (learned complexity), trained on 300K fast/slow samples

Efficiency & Deployment
- runs on 1,024 Ascend NPUs with Stale Synchronous Parallelism + prioritized queues
- training and inference run together using shared weights with no redundant copies
- vLLM-style decoder optimized for Ascend: chunked decoding, proactive prefetching, mitigates straggler delays

Benchmarks (System 2)
- GPQA: 68.0 (Qwen3-8B: 59.6)
- AIME24: 81.9 (GLM-4-9B: 76.4)
- LiveCodeBench: 67.1 (Nemotron-8B: 51.4)
- ArenaHard: 93.9
- MMLU-Pro: 79.0

Notable Design Choices
- data filtering via MinHash-LSH + ZIP algo for high diversity
- cold-start SFT with high-quality, multi-agent CoT data
- Zero-Advantage Mask avoids flat-gradient collapse.
- legal domain adaptation shows +4.6% accuracy over baseline

𝑷𝑨𝑷𝑬𝑹 𝑰𝑵 𝑨𝑳𝑻 ⮕ 𝑾𝑬𝑵 𝑴𝑶𝑫𝑬𝑳𝑺

[Kai Han @hankaixyz]

3/4 #ICML2025 SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
arXiv: https://t.co/UyasAfqGJY
#ICML @icmlconf

[Kai Han @hankaixyz]

2/4 #ICML2025 A new MoE architecture: Mixture of Lookup Experts
arXiv: https://t.co/qNmfVAreYj
#ICML @icmlconf

[Kai Han @hankaixyz]

1/4 #ICML2025 Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
arXiv: https://t.co/i8HH2gzuHO
#ICML @icmlconf

[Kai Han @hankaixyz]

From CoT to TOT, and now it's time for Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning https://t.co/i8HH2gA2xm

[fly51fly @fly51fly]

[CL] Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
Z Bi, K Han, C Liu, Y Tang... [Huawei Noah’s Ark Lab] (2024)
https://t.co/QzLFzAKOkF

[Ted Werbel @tedx_ai]

Researchers at Huawei Noah’s Ark Lab released Forest-of-Thought - an exciting new paradigm of post-training optimization to improve reasoning.

Brief introduction + breakdown of this recently released paper can be found on my blog (link in bio 👆)

[Kai Han @hankaixyz]

The scaling law of LLM reasoning
https://t.co/i8HH2gA2xm

[Jeff Dean @JeffDean]

@moderncpp7 @clu_cheng @NeurIPSConf @drfeifei @jiahuiyu @edchi I didn't see the talk, but the images I've seen of the slide seem quite offensive. Such generalizations should have no place in NeurIPS or anywhere else.