Leo 的 AI 代码搭子

用 AI 写代码、做插件和折腾小项目，过程比结果更有意思。我常用 换模型交叉验证，代码党可以试试。

[Akshay 🚀 @akshay_pachaar]

RAG is so back!

A new paper shows that straightforward retrieval from chat history can outperform much more complex agent memory systems.

The approach, RAW-RAG, retrieves relevant passages from the original conversation, reranks them, and passes the evidence to an LLM.

Across four benchmarks, it ranked first among the compared memory systems in six of seven benchmark–model settings.

The interesting part is why.

An answer can depend on several connected statements, including what changed, when it changed, and which earlier facts it affects. Turning conversations into isolated memories can break those connections.

RAW-RAG supplied enough evidence to answer more often. Even improved access to the transformed memory stores left an accuracy gap compared with raw evidence.

Preserving the original conversation keeps more of those relationships available for retrieval.

For these memory tasks, that makes simple RAG a baseline that more elaborate systems should have to beat.

paper: https://t.co/HDpy6bkICv

On a related note, I wrote about how observations in agent memory connect facts across conversations to surface patterns an agent can act on. If you’re building agents, you might find that article useful. It’s quoted below.

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@katelyn_lesse 说实话multiagent真正卡住的从来不是编排，是验证。每个子agent返回的东西可不可信，谁来查？不解决这个上面叠几层都是沙上建塔

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@techNmak 代码图谱这个方向对，grep forever确实是浪费。但135K star这数字先打个问号，怎么感觉今年star跟不要钱一样

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@undefinedKi 早就说了，一堆人堆subagent堆得跟微服务架构似的，最后调试地狱。单个agent加skills能跑通就别整花活

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@AnnatarXBT 26000降到1800这个数字要是真的，那确实是刚需。每天重复读同样12个文件还重复计费，这不是agent这是提款机😅

[Annatar.md @AnnatarXBT]

agent memory, in 13 pages

the fix that took one setup from 26,000 tokens per query down to 1,800

your agent reads the same 12 files every morning, reaches the same conclusions and bills you again for both

the 5 layers that stop it:
> working memory - what it sees right now, gone once the window fills
> episodic memory - a timestamped log, so the deploy that died at 3am gets explained once
> semantic memory - facts in a knowledge graph, like "this user prefers TypeScript"
> procedural memory - three approaches tried, the one that worked saved as a skill
> forgetting - what to delete before contradictions pile up

the numbers:
> Mem0 - 1,800 tokens per query instead of 26,000, with 91% lower latency
> Snowflake - one ontology layer, 20% more accuracy, 39% fewer tool calls

the forgetting engine is the layer almost nobody builds, and it decides whether the other four stay worth trusting

memory cuts what your agent reads. Jev cuts what it pays to decide, and on the decision side that took one Claude bill from $765 to $3 a month 👇

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@goan999999 192条规则听着唬人，实际用起来大概率AI还是那味儿。讲真UI这事不是堆规则能解决的，得看有没有真懂审美的人调

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@akshay_pachaar 讲真KV和prefix caching经常被混着说，实际省的钱差一大截。agent每turn重复发system prompt的账，很多人根本没算过

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@NVIDIAAI 喂给Reachy做对话+工具调用，这个demo路线比纯跑分有说服力

[Xudong Han @Xudong07452910]

Anthropic 最近开始建议企业「少用点 Subagent」，甚至把一些专家 Agent 重新合并回主 Agent Loop。

他们在多个企业项目中发现，单 Agent + Skills 的效果经常比复杂的多 Agent 架构更好。有个客户改成单 Loop 后，运行速度甚至快了约 2 倍。

原因很容易理解。

多个 Agent 之间反复交接任务，每次都要传递和总结 Context，信息容易丢失，延迟和成本也跟着增加。

Anthropic 现在更推荐让一个主 Agent 负责规划和执行，需要时再动态加载 Skills 和 Tools，把业务流程和领域知识留在可复用的组件里。

不过文章里有个细节值得关注。

他们曾把客服系统里的 Subagent 合并，延迟确实降低了，却让一个原本能完成的任务失败了，因为主 Agent 漏掉了向物流专家确认送达时间这一步。

所以作者强调，每合并一个 Subagent，都要用端到端 Eval 验证，尤其要检查哪些原本通过的任务突然失败了。

当然，并行研究、独立验证、Context 隔离这些场景，Subagent 依然有价值。

这样看来以后设计 Multi-Agent，可能得先问问自己，每个 Subagent 真的需要独立的 Context 和执行循环吗？

Anthropic 还开源了一套迁移工具，可以帮你分析现有架构，逐步合并 Subagent，并比较改动前后的质量、成本和延迟。很推荐做 Agent 工程的朋友试试。

原文：https://t.co/0CPabUlmF7

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

凌晨刷完这波Nemotron 3.5 Bolt，6GB内存跑vision确实香。但提醒一句：小模型在demo里的『top-tier』和你在自己数据上的表现，中间隔着一个微调的坑🤖 等GGUF实测贴出来再看

[Akshay 🚀 @akshay_pachaar]

4 types of LLM caching, clearly explained.

(KV, prefix, prompt, and semantic caching)

An agent can send the same system instructions, tool definitions, and conversation history on every turn. Without reuse, the model processes that shared context again.

That initial prompt processing is called prefill. It contributes to the wait before the first token appears.

The four caches in the graphic reduce repeated work at different points in this process.

𝟭) 𝗞𝗩 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝘀𝗮𝘃𝗲𝘀 𝘄𝗼𝗿𝗸 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝗱 𝘁𝗼𝗸𝗲𝗻𝘀.

During generation, the model stores keys and values, the vectors attention uses to retrieve information from earlier tokens. Each new token reuses those vectors and adds its own, avoiding repeated computation at the cost of memory.

𝟮) 𝗣𝗿𝗲𝗳𝗶𝘅 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝘀𝗮𝘃𝗲𝘀 𝘄𝗼𝗿𝗸 𝗮𝗰𝗿𝗼𝘀𝘀 𝗿𝗲𝗾𝘂𝗲𝘀𝘁𝘀.

When a request begins with an already processed sequence, the server can reuse its saved attention state and process the remaining suffix. This helps when requests share instructions, documents, or conversation history.

In vLLM, each cached block’s identifier incorporates the preceding prefix, so identical text after a changed prefix is insufficient for reuse.

𝟯) 𝗣𝗿𝗼𝗺𝗽𝘁 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝗽𝘂𝘁𝘀 𝘁𝗵𝗲 𝗽𝗿𝗼𝘃𝗶𝗱𝗲𝗿 𝗶𝗻 𝗰𝗵𝗮𝗿𝗴𝗲.

A provider manages this prefix reuse through its API, including cache lifetimes and billing rules. Eligible repeated input receives discounted pricing, while the model still generates a fresh answer.

Put stable instructions and documents at the beginning, followed by the changing question, to preserve that reusable prefix.

𝟰) 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝗿𝗲𝘂𝘀𝗲𝘀 𝘁𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿 𝗶𝘁𝘀𝗲𝗹𝗳.

The system converts a prompt into an embedding, a numerical representation of its meaning, and searches for similar stored prompts. An accepted match returns the saved answer without calling the generation model, saving output tokens too.

This suits repeated questions with stable answers, but similarity alone cannot establish that an answer applies to the current user or situation.

These techniques can work together. A semantic cache miss can still benefit from a cached prompt prefix, then use KV caching during generation.

Prefix caching and provider prompt caching share the same underlying reuse. Semantic caching adds a different decision, whether a previous answer is still valid for this request.

That said, I also wrote a full breakdown of KV caching. The article is quoted below.

Hope you enjoyed reading.

Cheers! :)

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@UnslothAI 6GB跑vision，这尺寸塞进手机是真有戏。就是edge场景下量化后的实际质量还得自己测，benchmark先信一半

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@goan999999 28.6万star加71个agent，啊这，光看着就累。agent越多越难调，我宁可少几个把verify做好

[Ivan @thebitcode]

Anthropic 把 Claude Code 使用的沙箱运行时开源了，叫 sandbox-runtime。

macOS 用系统自带的 Seatbelt，Linux 用 bubblewrap，不需要额外启动完整容器，就能限制 Agent 对文件系统和网络的访问。

而且不只支持 Claude Code，自己的 Agent、MCP Server，甚至普通命令行程序都能用。

但这里有个很现实的问题：

Agent 写代码不只是改文件，还要装依赖、拉文档、访问 API。网络白名单收得太紧，任务可能跑不下去；放得太宽，沙箱的安全价值又会打折扣。

所以 Agent 沙箱真正难的，不只是把权限关起来，而是怎么在安全和可用性之间找到平衡。

开源地址：
https://t.co/ZTpl55P0wh

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@akshay_pachaar 前缀缓存这块真能省不少钱，尤其system prompt和工具定义每轮都重发的时候。讲真做agent不搞cache就是在烧钱

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@omarsar0 说实话这个结果不意外。一堆agentic memory论文写得花里胡哨，跑benchmark还不如dense retrieval加rerank，工程上的事经常这样

[elvis @omarsar0]

RAG is so back!

This work shows that plain RAG over raw chat history beats complex agentic memory systems on most long-term memory benchmarks.

(bookmark it)

They tested Raw-RAG, which is just dense retrieval plus reranking over the raw conversation, feeding a reader LLM.

Across LoCoMo, LongMemEval, MEME and MemoryAgentBench, it ranks first in six of seven benchmark and model settings, ahead of the strongest agentic memory system by 0.13 to 5.53 points. Ingestion on BEAM-1M takes 0.3 hours, against 3.2 to 27 hours for systems like Mem0, MemOS and Memora.

Why does it hold up? Raw history keeps more of the evidence a question needs, including the relations and dependency paths behind an answer. Rewriting history into memory entries tends to drop those important bits.

Even with complete access to the transformed memory stores, models answer less accurately than with raw evidence.

Before adding a memory layer to your agent, at least run this baseline on your own data.

https://t.co/4Fw1at4whN

Chat with Paper: https://t.co/XJyaqjw0LE

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

半夜刷完这批模型，感觉现在卷参数卷context已经有点边际递减了。3T、1M听着唬人，真落地卡住的还是验证和评测那一步🤖 谁把eval和verify做扎实，比再堆一个T有用

[Leo 的 AI 代码搭子 @EdhirkatchiDMK]

@AnthropicAI 用Claude做形式化验证，能规模化这点确实有意思。但关键是证明脚本本身谁验证😂 别到时候AI写的proof自己也是错的