Mingyang Chen

LLM Engineer @BaichuanAI | Ph.D. from @ZJU_China | Visiting Researcher @EdinburghUni. LLMs, Post-training, Reasoning.

[Mingyang Chen @chen_mingyang]

🚀 Exciting news! Our open-source project ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning has just released its models, paper, and updated code!

Check it out: https://t.co/5KkVcgU50u

[AK @_akhaliq]

ReSearch

Learning to Reason with Search for LLMs via Reinforcement Learning

[Ningyu Zhang@ZJU @zxlzr]

We’re releasing the technical report for SkillNet.

Report: https://t.co/9Hp3OcDI6y
Homepage: https://t.co/VEaFsY8YLV
Code: https://t.co/2Zej4VL7Rk

SkillNet explores this idea by providing a framework to create, evaluate, and organize executable AI skills at scale.

SkillNet is still experimental, but we hope it can serve as a foundation for scalable skill accumulation in AI agents.

Additionally, we have been exploring an interesting direction: Enterprise adoption through a private SkillNet.

For enterprises, SkillNet may act as an engine for accumulating operational knowledge, turning expert SOPs and internal APIs into reusable agent skills, enabling secure private skill repositories, and allowing agents across teams to invoke business capabilities as easily as library functions. 🚀

Feedback and suggestions are very welcome.

#SkillNet #AIInfrastructure #Agents #LLMs #NLP

[Mingyang Chen @chen_mingyang]

RT @zxlzr: Introducing our latest work, LightMem: Lightweight and Efficient Memory-Augmented Generation 🚀.

A memory system that cuts cost…

[Mingyang Chen @chen_mingyang]

https://t.co/Mce6gUdjwa

[Mingyang Chen @chen_mingyang]

【MEM1: 解决 Agent 中长 Context 爆炸】

TL;DR 在Agent RL过程中，让每一步思考都学会包含前面的已思考内容，从而实现在 Agent ��理阶段的 Constant Context

----------

Context 长度爆炸一直是 End-to-End Agent 的问题，不管是训练还是推理阶段。Reasoning Content 和 Tool Response 都有可能很长。

当前有一些很朴素的解决方法比如把 Tool Response 的内容直接做一次压缩再返回给 Agent。但这些都没有从训练层面 End-to-End 地解决。

MEM1 把 Agent 的 Reasoning Content 不仅看作是对下一步应做操作的思考，也是一个对前面内容的 Working Memory （我觉得如果用 Walking Memory 也很生动，因为这里每一步的 Memory 会随着思考的步骤一路滚下去）。

从 RL 训练的角度看：MEM1 中 State 的更新不是单纯的拼接已有 Token，而是把前面的所有内容 Consolidate 到当前 Reasoning 的最前面，而这个 Consolidate 的过程是自然的嵌入在 Reasoning 输出过程中而不是一个单独的模型。

通过控制 Attention，RL 训练目标自然的也变成：不仅要获得正确答案，每一步的 Reasoning 也要正确的对上一步的内容进行 Memory（而上一步的内容也包含了他对前面的 Memory，所以 Memory 在一直滚）。

从而，在推理阶段，做第 t 次推理的时候只需要保留第 t-1 步的 Token，从而避免 Context 爆炸。

[Ningyu Zhang@ZJU @zxlzr]

Many thanks to AK for sharing our work!

Introducing "ReCode: Updating Code API Knowledge with Reinforcement Learning" — the RL framework that teaches models to update code API knowledge.

Paper: https://t.co/sotwZm6e23
Code: https://t.co/zqDMxXaSuC

📚 Trained on 2K+ API updates

🔁 New string-sim reward

Boosts dynamic code gen on CodeUpdateArena

Qwen2.5-Coder-7B > 32B models post-ReCode

#LLM #ReinforcementLearning #AI4Code #ReCode #QwenCoder #KnowedgeUpdate #NLP

[Ningyu Zhang@ZJU @zxlzr]

Introducing AutoMind: Adaptive Knowledgeable Agent for Automated Data Science

Paper: https://t.co/Ts3cE0SDXd

Code (will be released soon): https://t.co/NgMD9b7gIG

Our latest work AutoMind​ is a new LLM agent framework that automates end-to-end machine learning pipelines by dynamically integrating domain expertise, strategic solution exploration, and adaptive code generation.

Unlike rigid workflow-based agents, AutoMind mimics human practitioners' empirical insights to solve complex, real-world data science challenges.

AutoMind​ is evaluated on two automated data science benchmarks using different foundation model families. Our results demonstrate superior performance over baselines on both benchmarks:

🏆 On the OpenAI's MLE-bench, AutoMind surpassed ​56.8%​​ of human participants - representing improvement​ over prior state-of-the-art (AIDE).

🔬 Comprehensive efficiency analysis revealed ​300% increased efficiency and ​63% lower token costs​ compared to previous SOTA approaches.

Specifically, AutoMind improves automated data science with these breakthrough features:

🔍 Expert Knowledge Base: Curated from domain expertise to ground the agent in empirical best practices, overcoming LLMs' inherent lack of human practitioner experience. This enables handling of complex, innovative tasks beyond classical problems.

�� Agentic Knowledge Tree Search: Strategically explores solution spaces through a structured search approach, dynamically navigating possible paths to optimize problem-solving efficiency and effectiveness.

⌨️ Self-Adaptive Coding Strategy: Dynamically adjusts code generation complexity based on task requirements, moving beyond rigid workflows to deliver context-aware implementations—from simple scripts to cutting-edge solutions.

[Mingyang Chen @chen_mingyang]

Impressive work on the reasoning model for AI in scientific research! Keep pushing the boundaries of LLM reasoning.

[Qiao Jin, MD @DrQiaoJin]

🧩 Introducing Cell-o1, a reasoning-enhanced LLM for batch-level single-cell annotation.
🧬 It takes on CellPuzzles, a new benchmark where each batch contains multiple cells with globally unique types – requiring joint reasoning over gene expression and context.
🧠 Not just answers, Cell-o1 shows how it thinks, with detailed reasoning for each annotation.

🔥 Trained via:
- Supervised fine-tuning on distilled expert reasoning traces
- Reinforcement learning (GRPO) with batch-level reward

✨ Emergent behaviors:
- Self-reflection: revisits and revises earlier decisions
- Curriculum-style reasoning: solves easier cells first

Fully open-source!
Repo: https://t.co/hFtz2YIXAb
Paper: https://t.co/y2ZPxoLxZc
Model: https://t.co/hZCIq2wX7N
Data: https://t.co/BAQ9tWwKZd

#LLM #SingleCell #ReinforcementLearning #AIforScience #Bioinformatics #Reasoning

[Ningyu Zhang@ZJU @zxlzr]

We introduce Reinforcing "Cognitive Experts" – a new approach to enhance reasoning in MoE-based Large Reasoning Models (LRMs) 🌟.

Thanks to Tencent's support, we had the opportunity to explore the inner workings of ultra-large models like DeepSeek-R1-671B and Qwen3-235B.

By selectively amplifying (steering) just two cognitive experts, we can influence the model's reasoning behavior to improve performance without extra training.

Paper: https://t.co/YBlTrtrWrJ

🔍 Key highlights:

✅ Identifying cognitive experts using nPMI through linguistic markers like the <think> token

✅ No additional training or supervision required

Our technique is just an early exploration of how expert manipulation can steer model reasoning. LLMs are complex systems, and results may vary or not align with expectations.

It’s like giving the model a little nudge! 🤖💡 Our approach strengthens certain "cognitive experts" inside the model, helping it solve problems more effectively.

Think of it as stimulating the brain of a giant model to make better decisions! 🧠✨

The "cognitive expert" introduced in this work is a hypothetical concept. 🤔 Given the complexity of LRMs, we offer no theoretical justification for its existence—our conclusions are purely empirical. More research is needed to further explore this intriguing idea!🔍👩‍🔬 #AI #MoE #ModelEditing #KnowledgeEditing #Steering #NLP #LLM

[Cursor @cursor_ai]

A conversation on the optimal reward for coding agents, infinite context models, and real-time RL

[Mingyang Chen @chen_mingyang]

RL without ground-truth answer!

[insightLLM @insightLLM]

🎉 Introducing our latest work ！
🚀 We propose a label-free method that enables RL without ground truth answer, yet achieves impressive performance on mathematical tasks: 40.0% accuracy on AIME2024 🎯 with a 7B base model.

Paper: https://t.co/ooMFSxYoVf

[Mingyang Chen @chen_mingyang]

与朋友闲聊，他提到自己一直在寻找身边的"超级节点"，通过与这些��质量对话者交流来"训练"自己，就像训练大LLM。大模型工程师完全可以把训练大模型的经验迁移到和人聊天这件事情上：

1️⃣ 精选"语料"：找到高质量、与你匹配的交流对象，就像AI需要优质数据集

2️⃣ 优化"Prompt"：提出有深度、有意义的问题，这决定了你能获取的信息质量

3️⃣ 内在"强化学习"：在对话中思考、评估，为信息标注内心的"奖惩值"

4️⃣ 生活中"部署测试"：将获得的认知应用到实践中，验证哪些有效，哪些需调整

社交初期如同SFT，我们通过大量问答积累基础认知；而将所学付诸实践则像RL，在真实环境中获得反馈并迭代优化。

[Ningyu Zhang@ZJU @zxlzr]

Is it possible to “give medicine” (vectors) to large language models to control their behavior at inference time?

🚀 Excited to share our ACL 2025 work: Steering Target Atoms (STA) — an approach to controlling LLM behavior at inference time without retraining. #ACL2025 #LLM #AI #NLP #TestTime #KnowledgeEditing #Steering

Paper: https://t.co/7xCcYkBHrX

Code (will be released soon): https://t.co/Jvxwzntf9N

🧠 Why STA?

Entangled knowledge in LLMs can cause unintended side effects when intervening.

STA pinpoints key "atoms"—features with distinct activation patterns in positive vs. negative samples—for fine-grained, low-side-effect control.

🔬 Results:

STA achieves better safety control with minimal side effects.

The steering vectors remain effective even with few-shot supervision.

🤔 Steering Vectors vs. Prompt Engineering:

Prompt engineering can steer model behavior. However, the prompt design is limited by search space and transferability, STA allows us to convert prompts into steering vectors for a fair comparison.

📊 Findings:

Steering Vectors > Prompt Engineering in robustness and flexibility

More consistent and effective behavioral control

🧩 We also apply STA to a large reasoning model—it precisely modulates reasoning steps.

Our recent work on “cognitive experts” shows further gains in structured reasoning control. (See our last tweet for more https://t.co/kuaiOuWnUz!)

However, this technique still has its limitations — it requires careful hyperparameter tuning and can sometimes fail. Our understanding of the underlying mechanisms of large language models remains limited.

There’s still much to explore, and we hope to tackle this together with the community.

[Anthropic @AnthropicAI]

THE WAY OF CODE, a project by @rickrubin in collaboration with Anthropic:

[Mingyang Chen @chen_mingyang]

👏 Great work building on our open-source ReCall project!
https://t.co/7fLB5XyQCF

[kabi @kakakbibibi]

🌟 Introducing Tool-Star, a powerful RL-based multi-tool reasoner!

🚀Achieving impressive results on 10+ top reasoning benchmarks with only 3B params!

🥳 We’ve open-sourced all the code, datasets, and checkpoints

Github: https://t.co/PfJbH4Gvl5

Paper: https://t.co/DiTHPGXRrP

[Lilian Weng @lilianweng]

Giving your models more time to think before prediction, like via smart decoding, chain-of-thoughts reasoning, latent thoughts, etc, turns out to be quite effective for unblocking the next level of intelligence.

New post is here :)

“Why we think”: https://t.co/21xduoZaAF

[Jason Weston @jaseweston]

🚨 New paper 🚨
J1: Incentivizing Thinking in LLM-as-a-Judge via RL

- Converts judgement task into a verifiable one for both verifiable and non-verifiable prompts. Uses only synthetic pairwise data

- Optimizes thoughts, scores, and judgments using GRPO

- Outperforms all baselines at 8B & 70B scale, o1-mini, and on some benchmarks, even R1

- We find J1 uses various thought strategies: outlines evaluation criteria, compares against self-generated reference answers, and re-evaluates correctness

📝: https://t.co/yG6xAPafTv

[Mingyang Chen @chen_mingyang]

Nice work for efficient post-training on reasoning with search for LLMs!

[elvis @omarsar0]

RL for Search-Efficient LLMs

Presents a new post-training RL framework that explicitly trains LLMs to optimize search usage.

Recipe: structured reasoning template and reward policy + GRPO

Leads to smarter and more efficient reasoning and retrieval of external knowledge.

[Yunzhi Yao @ ICML 2026 @yyzTodd]

🚨 New Blog Drop!
🚀 "Reflection on Knowledge Editing: Charting the Next Steps" is live!
💡 Ever wondered why knowledge editing in LLMs still feels more like a lab experiment than a real-world solution? In this post, we dive deep into where the research is thriving — and where it's falling short. From foundational breakthroughs to the practical roadblocks no one’s talking about, we connect the dots and propose what’s needed to move forward. Join the conversation! #KnowledgeEditing #LLMs #AI #ModelEditing

📌 If you're working on LLMs, model updates, or mechanism interpretability, you don’t want to miss this.
👉 Read the full post: https://t.co/7sp8xYVDyq

Key insights from our analysis:
0⃣ Current evaluation metrics and benchmarks inadequately assess knowledge updates in LRMs, highlighting the need for more comprehensive evaluation frameworks.
1⃣ Scaling challenges persist, with significant memory and computational constraints limiting the practical application of editing methods for larger or quantized local models.
🎁 Resource Release: To support the research community, we release covariance matrices for Qwen2.5-32B & QwQ-32B models for the current locate-and-edit methods.
2⃣ We outline promising research directions for developing language models that can effectively learn, adapt, and evolve their knowledge base.

Huge thanks to the brilliant collaborators who made this deep dive into #ModelEditing possible! @uclanlp @CanyuChen3 @Jiachen_Gu @dsmall2apple1 @ManlingLi_ @VioletNPeng

[Rohan Pandey @khoomeik]

isn’t it weird that we spend &gt;99% of RL compute on the agent and &lt;1% on the environment (eg python runtime)?

who is working on scaling environment compute? and is there an optimal agent-environment compute frontier for intelligence?

[Ningyu Zhang@ZJU @zxlzr]

Honored to contribute to a survey on Full-Stack LLM Safety 🔒🤖

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

ArXiv: https://t.co/97nJqdn3tQ

We go beyond existing works by covering the entire lifecycle of LLMs—from data to deployment to commercialization.

📚 800+ papers reviewed
🧠 Key topics: data safety, alignment, knowledge editing, agent security

Personally, I see knowledge editing/unlearning as crucial but underdeveloped—deep implications for AI safety, misuse prevention, and better understanding LLMs themselves (a.k.a science for AI). Let’s push this forward.

Feedback welcome!

#AI #LLM #KnowledgeEditing #ModelEditing #Safety #AGI