Xingcheng Yao

LLM researcher @uclanlp, Prev @Kimi_Moonshot, @Tsinghua_IIIS, @princeton_nlp.

[Xingcheng Yao @StuartYao22139]

So big and so beautiful!! Really proud to be part of the team.

[Kimi.ai @Kimi_Moonshot]

🚀 Hello, Kimi K2! Open-Source Agentic Model!
🔹 1T total / 32B active MoE model
🔹 SOTA on SWE Bench Verified, Tau2 & AceBench among open models
🔹Strong in coding and agentic tasks
🐤 Multimodal & thought-mode not supported for now

With Kimi K2, advanced agentic intelligence is more open and accessible than ever. We can't wait to see what you build!

🔌 API is here: https://t.co/EOZkbOwCN4
- $0.15 / million input tokens (cache hit)
- $0.60 / million input tokens (cache miss)
- $2.50 / million output tokens

🔗 Tech blog: https://t.co/2RP7U3iakZ
🔗 Weights & code: https://t.co/4ukcXB0iP6
🔗 Github: https://t.co/B2bA4SfXBl
Try it now at https://t.co/85jA71X9gw or via API!

[Xingcheng Yao @StuartYao22139]

I'm grateful to be part of this moonshot spirit: Not because it's easy, but because it's hard.

[Xinyu Zhou @zxytim]

You see:
- a new arch that is better and faster than full attention verified with Kimi-style solidness.

I see:
- Starting with inferior performance even on short contexts. Nothing works and nobody knows why.
- Tweaking every possible hyper-parameter to grasp what is wrong.
- Trying to find the efficient chunkwise parallelizable form to squeeze juice out of the GPU
- RoPE or NoPE, a question haunting for nights.
- Fighting buggy implementation that causes one of the long-context benchmarks drops ~20 pts.
- RL diverging. Aligning training-inference numerics.
- Dedicated efforts to make sure comparisons are solid and fair.
- Going back-and-forth in a pool of adversarial gate-keeping tests, and finally it survives.

Great teamwork!

[Xinyu Zhou @zxytim]

You see:
- a new arch that is better and faster than full attention verified with Kimi-style solidness.

I see:
- Starting with inferior performance even on short contexts. Nothing works and nobody knows why.
- Tweaking every possible hyper-parameter to grasp what is wrong.
- Trying to find the efficient chunkwise parallelizable form to squeeze juice out of the GPU
- RoPE or NoPE, a question haunting for nights.
- Fighting buggy implementation that causes one of the long-context benchmarks drops ~20 pts.
- RL diverging. Aligning training-inference numerics.
- Dedicated efforts to make sure comparisons are solid and fair.
- Going back-and-forth in a pool of adversarial gate-keeping tests, and finally it survives.

Great teamwork!

[Xingcheng Yao @StuartYao22139]

More agentic part of K2 has been unleashed...

[Kimi.ai @Kimi_Moonshot]

Kimi K2-0905 update 🚀
- Enhanced coding capabilities, esp. front-end & tool-calling
- Context length extended to 256k tokens
- Improved integration with various agent scaffolds (e.g., Claude Code, Roo Code, etc)

🔗 Weights & code: https://t.co/SsFKTnWslD
💬 Chat with new Kimi K2 on: https://t.co/2bLWEHF6az
⚡️ For 60–100 TPS + guaranteed 100% tool-call accuracy, try our turbo API: https://t.co/EOZkbOwCN4

[Kimi.ai @Kimi_Moonshot]

Kimi K2-0905 update 🚀
- Enhanced coding capabilities, esp. front-end & tool-calling
- Context length extended to 256k tokens
- Improved integration with various agent scaffolds (e.g., Claude Code, Roo Code, etc)

🔗 Weights & code: https://t.co/SsFKTnWslD
💬 Chat with new Kimi K2 on: https://t.co/2bLWEHF6az
⚡️ For 60–100 TPS + guaranteed 100% tool-call accuracy, try our turbo API: https://t.co/EOZkbOwCN4

[Xingcheng Yao @StuartYao22139]

Check this out: https://t.co/qipE3yXKRv

[熊师傅 weight decay 了吗 @bigeagle_xd]

just finished the tech report and pushed to github. good night.

[Xingcheng Yao @StuartYao22139]

So excited!

[Arena.ai @arena]

🚨 BREAKING: @Kimi_Moonshot’s Kimi-K2 is now the #1 open model in the Arena!

With over 3K community votes, it ranks #5 overall, overtaking DeepSeek as the top open model.

Huge congrats to the Moonshot team on this impressive milestone! The leaderboard now features 7 different providers in the top 15 - the most competitive it’s ever been.

More insights in the thread 🧵

[Xingcheng Yao @StuartYao22139]

Machine learning technique really has no boundary, so does @jianlin_S Orz.

[jianlin.su @Jianlin_S]

QK-Clip: Taking Muon Further on the Scaleup Journey
https://t.co/HhKXmA3tbc

Interpreting the Key Training Techniques Behind Kimi K2: QK-Clip and MuonClip.

[Xingcheng Yao @StuartYao22139]

@JingyuanLiu123 海内存知己，天涯若比邻

[Xingcheng Yao @StuartYao22139]

@Grad62304977 I believe so. If not for MLA, QK norm will be working just fine

[Xingcheng Yao @StuartYao22139]

True. It's a Logit hard-capping with capping factors absorbed to the weight.

[You Jiacheng @YouJiacheng]

correction: actually there is a clamp_max on η.
(equivalently, rescaling only happens if max(qk) > t)

[Xingcheng Yao @StuartYao22139]

@YouJiacheng Great formalization! We'll make it clearer in the tech report.

[Xingcheng Yao @StuartYao22139]

@YouJiacheng yeah, it's counterfactual, but very useful.

[Xingcheng Yao @StuartYao22139]

@JingyuanLiu123 @zxytim 谢谢你让我想起来了big and beautiful在近期新闻里的含义🤣

[Xinyu Zhou @zxytim]

Meet Kimi K2, the BIG BEAUTIFUL MODEL that has the best agentic capability to-date.

We are curious what you could build with Kimi K2.

Proud of everyone in Kimi team. It's just a beginning. We'll keep building & keep shipping.

[Yining Hong @yining_hong]

Meet Embodied Web Agents that bridge physical-digital realms. Imagine embodied agents that can search for online recipes, shop for ingredients and cook for you. Embodied web agents search internet information for implementing real-world embodied tasks.

All data, codes and web environments are available at https://t.co/lJnWeUNlPC
Paper link: https://t.co/SOOpV4yzTa

[Xingcheng Yao @StuartYao22139]

@kaiwei_chang Congrats!

[Xingcheng Yao @StuartYao22139]

@teortaxesTex @YantaoLiu121668 @StringChaos @RotekSong I'm wondering too.

[Xingcheng Yao @StuartYao22139]

@zy27962986 @kellerjordan0 Thanks Zongyu!