Mo Li

Ph.D. candidate at THU @tsinghua_uni | INTP
Understanding the universe

[Mo Li @limoquan]

@FanqingMengAI can't agree more

[Mo Li @limoquan]

I'm just an Agent Loop.

[Mo Li @limoquan]

Vibe coding basically freed our hands, but at first, we still had to type prompts. Then voice input solved that—just speak and you're done. But now, we still have to use our eyes to read the results from Claude Code. It's actually super inconvenient and terrible for the eyes, especially staring endlessly at that high-contrast black-and-white terminal screen. I swear my eyesight has dropped recently just from looking at the screen too much. We desperately need an AI assistant or model whose sole job is to relay what Claude Code just did, to truly free our eyes.

[Elon Musk @elonmusk]

@_avichawla Impressive work from Kimi

[Kimi.ai @Kimi_Moonshot]

Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation.

Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.

🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.

🔗Full report:
https://t.co/u3EHICG05h

[CZ 🔶 BNB @cz_binance]

Tried many AI models with OpenClaw, I found Kimi AI to be the most token efficient, good at coding, also the easiest to set up.

[Kimi.ai @Kimi_Moonshot]

Kimi is currently the #1 on LLM Leaderboard (via OpenRouter)
This marks our first time topping the all-time usage charts, a huge milestone for the team. A massive thank you to every developer and user who made this possible. 🫡

[Mo Li @limoquan]

@ShengjieWa34067 You're spot on! I actually realized that right after posting. Just gave k2.5 a spin with 5 parallel sub-agents and it nailed the correct answer instantly. Works like a charm!

[Mo Li @limoquan]

🚀🚀🚀

[Kimi.ai @Kimi_Moonshot]

🥝Meet Kimi K2.5, Open-Source Visual Agentic Intelligence.

🔹Global SOTA on Agentic Benchmarks: HLE full set (50.2%), BrowseComp (74.9%)
🔹Open-source SOTA on Vision and Coding: MMMU Pro (78.5%), VideoMMMU (86.6%), SWE-bench Verified (76.8%)
🔹Code with Taste: turn chats, images & videos into aesthetic websites with expressive motion.
🔹Agent Swarm (Beta): self-directed agents working in parallel, at scale. Up to 100 sub-agents, 1,500 tool calls, 4.5× faster compared with single-agent setup.
-
🥝K2.5 is now live on https://t.co/YutVbwktG0 in chat mode and agent mode.
🥝K2.5 Agent Swarm in beta for high-tier users.
🥝For production-grade coding, you can pair K2.5 with Kimi Code: https://t.co/A5WQozJF3s
-
🔗 API: https://t.co/EOZkbOwCN4
🔗 Tech blog: https://t.co/6h2KkoA0xd
🔗 Weights & code: https://t.co/H38KegeDIY

[Mo Li @limoquan]

I'm envisioning a 'Pre-commit Hook' for academic papers, but powered by AI Agents instead of rigid code. Traditional scripts are too brittle for writing, but an Agent could handle the nuance: detecting citation hallucinations or spotting semantic slips (like accidentally swapping 'Type A' for 'Type B' mid-paragraph). AI naturally excels at these fuzzy boundary errors. It’s time to move from simple linting to full Agentic CI/CD for research. #AI #AcademicWriting"

[Mo Li @limoquan]

@rach_it_ @Meta @KempnerInst @astonzhangAZ @rish2k1 @Devvrit_Khatri @dvsaisurya @louvishh @brandfonbrener @elmelis Fascinating work on qTTT! The theoretical analysis on the log margin requirement and the FLOP-matched comparison against 'thinking' tokens are super insightful.

Is there any plan to release the code? Would love to explore this further for retrieval tasks.

[Mo Li @limoquan]

Solid read on "VL Norm" (ICLR 2026 submission). They tackle RLVR gradient instability caused by dynamic response lengths (L).
While their MVUE theory suggests alpha=1 (1/L) is optimal, experiments (Table 3) show alpha in [0.5, 0.75] is highly competitive.
My hypothesis: If we view gradient summation as a high-dim Random Walk, the norm might scale with sqrt(L) (Central Limit Theorem). Normalizing by sqrt(L) (alpha=0.5) could potentially decouple the update step size from sequence length, keeping the "push" constant.
Alpha=1 minimizes noise but might risk signal dilution. Perhaps the sweet spot balances Variance Reduction vs. Signal Preservation? ⚖️

[Rosinality @rosinality]

ΔL Normalization: Rethink Loss Aggregation in RLVR

Better loss normalization for dealing with different response lengths.

[Mo Li @limoquan]

Interesting work

[Rohan Paul @rohanpaul_ai]

This paper says a language model hides many internal policies, and training them bottom up improves reasoning.

On Qwen3, 4B model, Bottom up Policy Optimization (BuPO) raises the average score by 3.43 points.

A policy here means the rule that turns the current text into probabilities for the next word.

A Transformer stacks layers, and each layer uses self attention to read earlier words plus a feed forward block to add learned patterns.

The paper turns each layer's internal summary into next word probabilities, so every layer becomes a real internal policy.

They also isolate what attention adds versus what the feed forward block adds, so the roles are separated.

They track entropy, how spread out the guesses are, and they see early layers explore while late layers narrow down.

Llama stays broad until the end, but Qwen3 tightens step by step, and BuPO trains lower layers early so later training starts from better guesses.

----

Paper Link – arxiv. org/abs/2512.19673

Paper Title: "Bottom-up Policy Optimization: Your Language Model Policy Secretly Contains Internal Policies"

[Mo Li @limoquan]

Some thoughts on current LLM memory systems (MemGPT, Mem0 etc.): they feel like transitional solutions—engineering "patches" for current architectural limitations.

From an information theory perspective, any external summary is essentially lossy compression; the ideal upper bound of memory is simply Full Context. We currently rely on external modules mainly because standard Attention's KV cache growth makes infinite context computationally prohibitive.

However, architectures like Linear Attention or SSMs (maintaining a fixed-size state) point toward a future where inference remains O(1).

It reminds me of the pre-deep learning era in Computer Vision. Much like how manual feature engineering was replaced by end-to-end learning, I suspect current "Memory Engineering" (manual storage/retrieval logic) will eventually be superseded by native, end-to-end learned memory patterns within the models themselves.

[Rohan Paul @rohanpaul_ai]

REPO, short for context repositioning, lets a language model learn better token positions in messy long inputs.

It adds 0.9% parameters yet boosts noisy context scores by about 11 points.

Transformers use attention, the part that decides what earlier tokens matter, but it needs position labels to know order.

Most models just label tokens 0, 1, 2, and so on, which can hide what matters in noise.

The paper links this to cognitive load theory, an idea that working memory is limited, so bad layouts waste capacity.

REPO adds a tiny module that reads each token's internal state and predicts a real number position per attention head.

Those predicted positions go through the usual position encoding function, so distant but relevant tokens get more attention.

So even with lots of filler, it can still focus on the key line.

They trained it on the OLMo-2 1B base model and report gains on noisy text, tables, and long contexts, with normal short task performance.

It keeps the normal left to right generation order, so speed stays close to the original model.

----

Paper Link – arxiv. org/abs/2512.14391

Paper Title: "RePo: Language Models with Context Re-Positioning"

[John Wen03 @JohnWen03]

@iclr_conf The leak severely undermines academic fairness. The loss of reviewer anonymity discourages critical feedback. Authors gain potential for off-the-record lobbying, violating equitable debate. The credibility of all acceptance decisions is fundamentally questioned.

[Mo Li @limoquan]

Single-shot APIs as we know them will end.
As Ilya Sutskever said, “Pre-training as we know it will end.” In the same spirit, I believe the basic “send a prompt, get a response” LLM API will be replaced by agent APIs: you call an autonomous agent with a goal, and it internally handles tools, retries, exceptions, and long messy workflows. The API surface itself becomes the agent, not just a single request–response function.

[Kimi.ai @Kimi_Moonshot]

🚀 Hello, Kimi K2 Thinking!
The Open-Source Thinking Agent Model is here.

🔹 SOTA on HLE (44.9%) and BrowseComp (60.2%)
🔹 Executes up to 200 – 300 sequential tool calls without human interference
🔹 Excels in reasoning, agentic search, and coding
🔹 256K context window

Built as a thinking agent, K2 Thinking marks our latest efforts in test-time scaling — scaling both thinking tokens and tool-calling turns.

K2 Thinking is now live on https://t.co/YutVbwktG0 in chat mode, with full agentic mode coming soon. It is also accessible via API.

🔌 API is live: https://t.co/EOZkbOwCN4
🔗 Tech blog: https://t.co/n7xxaszqzF
🔗 Weights & code: https://t.co/4ukcXB0iP6

[Mo Li @limoquan]

Allowing subagents to call other subagents is like starting a nuclear chain reaction — it could spiral out of control.
As Oppenheimer said, “Now I am become Death, the destroyer of worlds.”

[Mo Li @limoquan]

By actively "sculpting" its context (folding, summarizing, searching), the LLM can filter out noise and focus.

And it works! We're seeing great results on tough Long Context benchmarks.

Check out the paper: "Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management"
https://t.co/ewfIf7jkCt

#LLM #AI #LongContext #ContextEngineering #Sculptor #AIResearch

[Mo Li @limoquan]

Ever feel that as LLM context windows get longer, the models actually get... dumber? 😅

They suffer from "Lost in the Middle" (forgetting info) or get confused when old info interferes with new tasks.

It's like a Working Memory buried under piles of old files! 😫

[Mo Li @limoquan]

Introducing our new work: Sculptor! 🎨

Instead of just building a bigger, messier workspace, we teach the AI to actively manage its own context.

We give the LLM a toolkit 🧰 for "Active Context Management" (ACM), treating it as a cognitive agent that can decide what to focus on.