Cooper Leong

PhD student @HongKongPolyU
LLM Mechinterp/Safety

[Cooper Leong @cooperleong22]

Explore LLM Interpretability with this comprehensive resource compilation:
https://t.co/eVliOHNUip
📚 Tutorials, libraries, surveys, papers, blogs & more!
📂 Categorized for easy navigation
🔄 Continually updated
🗨️ Your thoughts & feedback are welcome!
#NLProc #LLM

[Z.ai @Zai_org]

Introducing GLM-5.2: Frontier Intelligence, Open Weights

- Significant improvements in coding and agentic tasks
- Strong long-horizon capabilities with a 1M context window
- Two levels of reasoning effort: GLM-5.2 (max) pushes the limits, while GLM-5.2 (high) strikes a strong balance between performance and token efficiency
- MIT-licensed open weights
- Same API pricing as GLM-5.1

Tech Blog: https://t.co/LAsxUdN0JZ
Weights: https://t.co/g0A1C4UWx4
API: https://t.co/Kc3E22cbN7
Coding Plan: https://t.co/Nk8Y98HNhU
Chat: https://t.co/WCqWT0qCQb

[Cooper Leong @cooperleong22]

This is the scenario in which only one country claims to have developed the world’s first nuclear weapon and immediately signed the Nuclear Non-Proliferation Treaty.

[elie @eliebakouch]

mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community

also the fact that this is un purpose not visible to the user is crazy

[Rosinality @rosinality]

https://t.co/968qcCMAKQ

With respect to the degree of update on-policy distillation stays between RLVR and SFT. For the trajectory OPD remains a low-rank update from early on (maybe because it is distillation anyway). Could this affect the generalization?

[Lisan al Gaib @scaling01]

Claude Mythos speeds up training code of small AI models by 52x

Humans need 4-8 hours to reach a 4x speedup

[Cooper Leong @cooperleong22]

we might have a new reset...again

[Cooper Leong @cooperleong22]

@thsottiaux auto session latest summary and title for cli. currently, if we fork several sessions from the same prefix, they all show the same prefix and are only distinguished by different update times.

[RyanLee @RyanLeeMiniMax]

Recently, we took time to consolidate all of the work behind M2 and published it here: our M2 paper on arXiv

It’s been just over six months since we first open-sourced M2 on December 23 last year.
During that time, a number of our ideas and systems have been broadly adopted by the open-source community — including CISPO, Forge RL System, Self-Evolution.
Over the past six months, we’ve felt incredible enthusiasm from the open-source community. Nearly every model release reached the #1 spot on the Hugging Face leaderboard.

Now it’s time for a new chapter.
We’re getting ready for M3.

MSA paper is on the road.
https://t.co/jeLPMhtuIx

[Ant Ling @AntLingAGI]

SwiGLU is everywhere in modern LLMs — but for large inputs it behaves like x². That quadratic blow-up inflates activations, amplifies outliers, and makes deep network or low-precision (FP8/FP4) training prone to loss spikes.

We propose PowLU, a drop-in activation built for stable large-scale pre-training. 🧵

[Weiwei Sun @sunweiwei12]

Excited to share our new work on Reinforcing Human Behavior Simulation via Verbal Feedback.

Can human simulators learn from feedback, not just rewards?

Most RL for LLMs turns feedback into a single score. But human behavior is rarely just right or wrong. It is social, contextual, subjective, and multi-dimensional.

A score can tell the model what is better. Verbal feedback can tell it why.

Meet DITTO + SOUL.

Paper: https://t.co/G0cEHr53h0
Code: https://t.co/6osJizwUDi
Model: https://t.co/yIAvpbKPSd

[Binfeng Xu @billxbf]

Excited to release 🌟Polar🌟, our Agent RL rollout infra for real-world harnesses. Be it Codex, Claude Code, OpenClaw, Hermes, or your self-made ones 🔥 -- Polar takes your harnesses directly as training environments without code change.

Find a problem, design the harness, and train your own agents! 🧵

[Cooper Leong @cooperleong22]

👀

[Andrej Karpathy @karpathy]

Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.

[Cooper Leong @cooperleong22]

@flyingpetal472 @YouJiacheng app里的模型在基座基础上还有后训练

[Tibo @thsottiaux]

We found and fixed two issues that could explain this degradation of the capability of GPT-5.5 in Codex over the last ~ 48 hours.

We are monitoring over the coming hours to fully confirm and I will reset usage limits this evening.

Apologies and now is the time for /fast maxxing.

[Cooper Leong @cooperleong22]

👀

[Tibo @thsottiaux]

Codex team is aware of reports of GPT-5.5 performing worse for some users and investigating. We don't have anything conclusive yet and systems are healthy but we will share updates as we go.

[Cooper Leong @cooperleong22]

The hard part isn’t doing the right thing.
It’s making fewer mistakes while staying sustainable across longer time horizons and larger scales — in data, infrastructure, and life.

[Will Held @WilliamBarrHeld]

To train better open models, we need predictable scaling.

Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.

Getting there took some work 🧵

[Lee Sharkey @leedsharkey]

My team at @GoodfireAI has been cooking up a new way to do interpretability: decompose a language model’s weights, not its activations.

Our decomposition natively handles attention (!) and behaves less like a lookup table and more like a generalizing algorithm. (1/6)

[Cooper Leong @cooperleong22]

The fact that a token embedding is trained to contain information about this token is questionable if you think that the goal of feeding this is to predict the next token.

[Muyu He @HeMuyu0327]

I am now highly skeptical of the claim that adding the token embedding to deeper layers improves the model by "preserving the original token information", and think that the reason it improves at all is much simpler.

How the hypothesis was made. It was proposed in the Value Residual Learning Paper based on the **fact** that if you add the first value vector v1 / token embedding x0 to deeper layers' value vector / residual stream with equal weight (0.5 * v1 + 0.5 * v), the model's validation loss improves significantly. And we later found that adding **any** linear transformation of x0 helps just as much.

Ablation setup. If the model truly improves because deeper layers have access to the original x0 information, then this ablation should not change the model performance: killing the gradient of x0 when it is added to subsequent layer in this extra path.

Since x0 (and the embedding layer) will receive regular updates via the standard computation path, x0 will always be able to supply token information to deeper layers, and deep layers' attention module can learn to use it properly.

Therefore, for both adding x0 to later x and adding a linear transformation of x0 to later v, we run an ablation that detaches/kills x0's gradient during the forward pass.

Experiment result. In both ablations, we find that most of the improvement is gone. Although the model has access to a perfectly valid x0 information in deeper layers and can update attn/MLP weights to utilize it, it never recovers most of the benefits we see in the baseline.

This seems to suggest that "value residual learning" mostly (not all) works not because valuable x0 info is passed down, but because there is some benefit to **the embedding layer** by adding x0 to deeper layers.

There might be two ways the embedding layer can be benefitted. One is just pure gradient benefits: that value residual learning is some advanced form of residual connection that handles vanishing gradients better. Need to do some math to see if this holds. The other is that the forward pass set up in this way updates the embedding space in a meaningful way, so tokens can have a more optimal representation.

Next up will want to do ablations to test both hypotheses. And of course I might just have missed something simple.

[Yanshu Li✈️ICML2026 @karrsen0713]

Accepted at ICML 2026! Take Action for a Better RLVR!

[Cooper Leong @cooperleong22]

👀

[Xiaoyin Qu @quxiaoyin]

I can’t believe I stopped using Claude Code max and entirely use DeepSeek and Hermes. It’s so fast, so so fast, 3x faster for the same task. So cheap. I spent $5 last week and never need worry about being rate limited or usage hit limits very two hours. For most tasks it’s perfect enough.