Yaowei Zheng | LlamaFactory

70k ⭐️ OSS @llamafactory_ai, CTO @prismshadow_ai, ex-@ByteDanceSeed_
Building PenguinHarness - Best Self-Improving Harness. Opinions are my own.

[Yaowei Zheng | LlamaFactory @code_hiyouga]

👋 Hi everyone! Long time no see

We're the LlamaFactory team. After passing 70k stars on GitHub, we're now building the "LlamaFactory" of recursive self-improvement (RSI), towards a unified, efficient, stable, and flexible framework for agent self-evolution. Excited to share our team's latest work with you all.

TL;DR:
🐧 We've open-sourced PenguinHarness, a multi-agent self-evolution framework that brings every RSI algorithm under one roof, over 2,600 GitHub stars.

📖 To get a fuller picture of existing RSI work, we went through 140 papers, organized and categorized them into the awesome-rsi repo, and published a survey paper.

📊 To compare self-evolution efficiency across algorithms fairly, we built GDPevo, a self-evolution benchmark covering 6 enterprise scenarios and 240 tasks, and used it to evaluate 6 mainstream agents.

The details:

Following the same philosophy as LlamaFactory, we want to bring all RSI-related data, models, algorithms, and benchmarks into a unified open-source harness, so everyone can try and debug RSI more easily. We've already integrated 5 RSI algorithms and 7 benchmarks, and we're aiming for 100+ down the road.

Borrowing from DSH's "everything is a plugin" approach, we've packaged every algorithm and benchmark as a plugin. They're fully decoupled, so users can freely mix and match any RSI algorithm to evolve different agents. We also built a modern desktop client that installs in one click on any OS and runs locally, so you can grab the latest version from our website.

Website: https://t.co/CvMVlaA6Kj
OSS Repo: https://t.co/OYXXTp50PL

To better understand where the academic work stands today, we open-sourced the awesome-rsi repo and published a survey paper covering 140 recent works. We also mapped the trajectory of RSI against that of deep learning — many concepts line up one-to-one, only now the subject is an agent.

OSS Repo: https://t.co/DpjwtGnqcg
Paper: https://t.co/DMJNFIPUxJ

To better evaluate how different harnesses and methods perform at self-evolution, we built the GDPevo benchmark, spanning 6 real-world enterprise scenarios and 240 tasks. With a train/val split, it fairly measures an agent's self-evolution ability while guarding against overfitting.

OSS Repo: https://t.co/Eiwlc9ExOD
Paper: https://t.co/cYINn4R8A0

This post isn't just about our work. We're also hoping to invite friends from academia and industry to join our open-source project. If you'd like to see your name on an open-source project and co-author papers with us, DM us and let's talk!

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@guochen622 千万别这么说，之后多交流

[Yaowei Zheng | LlamaFactory @code_hiyouga]

👋 Hi everyone! Long time no see

We're the LlamaFactory team. After passing 70k stars on GitHub, we're now building the "LlamaFactory" of recursive self-improvement (RSI), towards a unified, efficient, stable, and flexible framework for agent self-evolution. Excited to share our team's latest work with you all.

TL;DR:
🐧 We've open-sourced PenguinHarness, a multi-agent self-evolution framework that brings every RSI algorithm under one roof, over 2,600 GitHub stars.

📖 To get a fuller picture of existing RSI work, we went through 140 papers, organized and categorized them into the awesome-rsi repo, and published a survey paper.

📊 To compare self-evolution efficiency across algorithms fairly, we built GDPevo, a self-evolution benchmark covering 6 enterprise scenarios and 240 tasks, and used it to evaluate 6 mainstream agents.

The details:

Following the same philosophy as LlamaFactory, we want to bring all RSI-related data, models, algorithms, and benchmarks into a unified open-source harness, so everyone can try and debug RSI more easily. We've already integrated 5 RSI algorithms and 7 benchmarks, and we're aiming for 100+ down the road.

Borrowing from DSH's "everything is a plugin" approach, we've packaged every algorithm and benchmark as a plugin. They're fully decoupled, so users can freely mix and match any RSI algorithm to evolve different agents. We also built a modern desktop client that installs in one click on any OS and runs locally, so you can grab the latest version from our website.

Website: https://t.co/CvMVlaA6Kj
OSS Repo: https://t.co/OYXXTp50PL

To better understand where the academic work stands today, we open-sourced the awesome-rsi repo and published a survey paper covering 140 recent works. We also mapped the trajectory of RSI against that of deep learning — many concepts line up one-to-one, only now the subject is an agent.

OSS Repo: https://t.co/DpjwtGnqcg
Paper: https://t.co/DMJNFIPUxJ

To better evaluate how different harnesses and methods perform at self-evolution, we built the GDPevo benchmark, spanning 6 real-world enterprise scenarios and 240 tasks. With a train/val split, it fairly measures an agent's self-evolution ability while guarding against overfitting.

OSS Repo: https://t.co/Eiwlc9ExOD
Paper: https://t.co/cYINn4R8A0

This post isn't just about our work. We're also hoping to invite friends from academia and industry to join our open-source project. If you'd like to see your name on an open-source project and co-author papers with us, DM us and let's talk!

[Ben Shi @BenShi34]

To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.

This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.

All data stays local, unless you choose otherwise. Lmk how it is!
Blog: https://t.co/ad3hFmqHls
Link to download: https://t.co/i9P7mIlfK6

[MiniMax (official) @MiniMax_AI]

MiniMax M3.1 Flash: Space Performance + Bunny Speed

🚀🐰 Yes, Space Bunny - which has topped global leaderboards with 200T+ tokens - was an earlier version of M3.1 Flash.

💰Redefining the performance economics of agentic AI. M3.1 Flash achieves frontier-level intelligence across key tasks such as Cowork (Automation-Bench: 62%; ALE: 30%), Coding (DeepSWE: 74%) and Visual Reasoning (BabyVision: 91%), at 1/3 the cost of M3 and 3x the speed.

API pricing (flat rate): $0.15 / $0.60 per 1M input/output tokens, with cache reads at $0.015 / 1M tokens.

Model weights will arrive on Oct 23.

API: https://t.co/fHRdSV73Hr
M Plan: https://t.co/XfJWSyauda
MiniMax Code for desktop: https://t.co/SGmLTI2HQm

[Wenhao Chai @wenhaocha1]

New Blog: Towards the Agentic Era of Video Understanding

For complex tasks, agents have become the default. Nobody uses chat mode for that work anymore: people hand it to Claude Code, Codex or Antigravity (yes sometimes🤣), which open files, run commands and check their own results. We think video understanding should benefit from this too.

So we are releasing Video-Index 1.1 with an agentic evaluation setup (which should be by default!). more in the threads🧵

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@huangruiteng The idea of "context slope" is really interesting.

[DeepSeek @deepseek_ai]

🚀 DeepSeek-V4.1-Flash-1017 is here!

Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.

🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.

1/4

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@AstroHanRay 感谢推荐！

[Yifan Xu @yifanxu_ephai]

这是我正在做的 DSH-LayerFS。

https://t.co/1mLJvxj6Xu

基于 LayerFS 的存算隔离架构和存储压缩能力，做了一个极简的 multi-agent harness：

- Agent 的每一步执行都有记录可追溯
- 每条 Bash 命令都在独立的 Workspace 中运行
- 内置冲突检测机制
- 支持递归嵌套 Fork（Recursive Nested Forking）

再配上这么可爱的鲨鱼娘视图——

大家愿意再给 DSH 一次机会吗？🦈

@tianyi

[Shangyin Tan @ShangyinT]

Autoresearch with coding agents can now run for days on a single problem, but we've seen an increasing number of cases where they stop improving early. We address this issue with a simple idea: forgetting.

Introducing our new paper: "Scaling Discovery Requires Forgetting"

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@eggbaby Thanks buddy！

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@EthanWang07X 感谢感谢🫶

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@bbruceyuan 非常感谢👏👏

[Yuu💖 @QuantumTransf]

还是来正式宣传一下，感觉今天 gosim 有点虎头蛇尾（悲

我们做了 obelisk 这个项目，作为提供给 coding agent 检索自己 session 能力的一个 infra，同时也让人可以很好的审计 coding agent session

仓库：https://t.co/TJlm8kVuh9

Q1：为什么市面上有那么多 memory 产品，我们仍然需要一个这样的 infra？

A1：
1. 因为很多 memory 都是黑盒产品，raw session 仅在提取 memory item 时，在后续需要用到这些提取产物时，agent 没有回查原始 session 的意愿和能力，它甚至不知道被注入上下文的 memory 的来源是什么
2. 因为被提取的 memory item 是有其 scope 的，而这些 scope 并不被 agent 使用时所知，agent 会扩大这些结论，从而失去准确性；并且我们也不应该认为被提取出来的内容能够真的对现在有效，因为现在需要什么是过去所无法知道的
3. 这样的一个 infra 使得使用 history 成为一个完全透明的过程（对人和 agent 都是），代价是 search 的成本被转移到了 agent 的上下文窗口里，但我们可以使用 subagent 来解决这个问题；它本质就是为 agentic research 提供了材料——历史 session 内容
4.我们做了一些 performance 的测量，在一些 benchmark 上的表现是优于 mem0 的，这部分可以说明它不劣于传统 memory；我们还在寻找更有力的证明

Q2：你们是如何构建 session 检索层的？
A2：
这个问题说简单也不简单，底层是普通数据库选型，没有任何魔法——对，我们放弃了向量匹配，也就不需要向量数据库
我们之前在一篇文章里详细阐述过为什么我们只需要 bm25 / fts5 这种精确匹配，可以查看 https://t.co/gnDAoZVNNR，简单来说一方面就是向量匹配有模糊性，而我们不想要模糊性，另一方面是，在 agentic search 中 one shot 的召回率并没有那么重要，但是准确率很重要，召回错误的 item 比不召回任何东西的 negative effect 影响更大
并且我们使用了 codemode 作为面向 agent 的 interface，这使得 agent 可以很好组合各项查询，以及控制上下文长度，这里面有一些 trick

Q3：我想知道 agent 在这个 infra 上的 performance 如何？它是否只能处理一些简单的任务？比如这个代码之前为什么这样写？
A3:
目前我们试过一些非常有挑战性的任务，这其实完全取决于 agent + model 能力，越强表现越好；我们尝试过简单的 memory 任务，让 agent 看看这个项目的代码为什么这么写，让 agent 获取另一个 task 的进度，或者做 handoff，也试过难一些的，让 agent 写周报，或者画出项目 roadmap（搭配 Claude Artifact 效果更佳），甚至更长期的，对于项目选型，架构，coding 做长达一两个月的复盘（请见上面的文章内容）
这些产物都不是简单的 memory item 能够涵盖的，我相信这是一个 aha moment

如果还有其他 qa 也欢迎~

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@FanqingMengAI Thank you Fanqing. Let's go further on RSI 🫡

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@Chonghan_Liu thanks bro

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@konghayao 谢谢！希望Peri Code也越来越强

[Yaowei Zheng | LlamaFactory @code_hiyouga]

@yifanxu_ephai Thank you very much!

[Yaowei Zheng | LlamaFactory @code_hiyouga]

大家好！很高兴分享我们团队的开源新作 PenguinHarness～

在 Github 获得了 7 万 Star 以后，我们正在打造自进化（RSI）领域的 LlamaFactory，志在成为统一、高效、稳定、灵活的 Agent 自进化框架。

一句话介绍：
🐧 我们开源了 Multi-Agent 自进化框架：PenguinHarness，将所有 RSI 算法集成到一个统一框架中，现已获得了 2600 Github Stars。

📖 为了更全面地了解现有 RSI 工作，我们详细调研了 140 篇论文，将它们归纳整理分类，形成了 awesome-rsi 仓库，并发表了综述 paper。

📊 为了公平对比不同算法的自进化效率，我们构建了自进化 Benchmark：GDPevo。涵盖 6 大企业场景、240 个 Task，对 6 种主流 Agent 进行了评测。

以下为详细介绍：

和 LlamaFactory 的思路一样，我们想把所有和 RSI 有关的数据、模型、算法和评估都集成到一个开源 Harness 框架里面，方便大家实验和调试。现在已经集成了 5 种 RSI 算法和 7 个 Benchmark，未来我们的目标是扩充到 100 多个。同时，我们借鉴 DSH 的 Everything is Plugin 思路，把所有的算法和 Benchmark 都做成插件，相互解耦，让用户可以灵活调用任意 RSI 算法进化不同的 Agent。我们也做了精美的客户端，任何操作系统都可以一键安装，本地运行，可以到官网下载最新版体验。
项目官网：https://t.co/CvMVlaA6Kj
开源仓库：https://t.co/OYXXTp50PL

为了对现有学界工作有更好的理解，我们开源了 awesome-rsi 仓库，并且发表了综述论文，对近期的 140 篇工作进行了梳理，还把 RSI 的发展路线和深度学习的路线进行了对照，发现很多概念都是一一对应的，只不过现在的对象变成了 Agent。
开源仓库：https://t.co/DpjwtGnqcg
论文链接：https://t.co/DMJNFIPUxJ

为了更好地评估不同 Harness 与方法的自进化效果，我们做了 GDPevo Benchmark，涵盖 6 大真实企业场景，共 240 个任务。通过进行 Train-Val 划分，在防止过拟合的前提下公平评估 Agent 的自进化能力。
开源仓库：https://t.co/Eiwlc9ExOD
论文链接：https://t.co/cYINn4R8A0

这条帖子不仅仅是介绍我们的工作，还希望能号召各位学界、工业界的伙伴来参与我们的开源项目。如果你也想在开源项目署名，共同发表论文，欢迎私信详聊～