Jimmy

Building data system school for software engineers.
Building games for fun.

[yishan @tspy]

https://t.co/7RRfhyg36u

[Diyi Yang @Diyi_Yang]

This fall @michaelryan207 @jyangballin and I are teaching a new course CS329Z "Engineering AI Agents" on how to build AI Agents from scratch. Come join us and learn how to build them 🤖

[思维怪怪 @0xLogicrw]

前 MIT 教授、Reflexion 共同作者 Ashwin Gopinath 提出一个猜测：最近在硅谷爆红的 个人 Agent 助手 Instinct 真正和别家 Agent 拉开差距的，可能不是执行能力，也不是普通的长期记忆，而是对用户「长期状态」的持续维护。

Gopinath 曾是创始人 Noah Shinn 的老师，两人 2023 年共同参与 Reflexion 研究。他没有 Instinct 的内部信息，以下判断来自实际使用体验：

单看任务执行，Instinct 并没有比 Hermes、OpenClaw 等 Agent 多出什么惊人的能力。现在很多 Agent 也已经会复盘、保存记忆甚至自动生成 Skill。Instinct 更特别的地方，是使用时间越长，似乎反而越了解用户，没有明显被越来越多的历史信息拖累。

关键可能在于，它没有简单保存全部聊天和操作记录，再靠向量搜索或知识图谱找回来，而是在持续维护一份「当前状态」：哪些用户偏好仍然有效，哪些事实已经变化，哪些任务还没完成，下一步应该等、继续跟进，还是直接行动。

这对长期 Agent 很重要。一个持续几天甚至几周的任务，中间可能经历邮件回复、等待、条件变化和多次跟进。Agent 不只要记住过去发生过什么，还要知道哪些信息已经过时，现在真正重要的是什么，以及什么时候根本不需要打扰用户。

Gopinath 把这种过程称为「记忆编译」：不断从杂乱的生活记录里筛出有用信息，压缩成一份持续更新的状态。重点不是「记得更多」，而是知道什么值得留下、什么应该失效，以及什么变化值得触发行动。

这和三年前的 Reflexion 有一条明显的延续线。Reflexion 让 Agent 把失败总结成文字记忆，用于下一次尝试；到了长期个人 Agent，问题进一步变成了如何持续理解一个不断变化的人和现实世界。

大模型能力越来越接近，工具调用和执行框架也在快速普及，真正难复制的部分，反而可能是谁能长期记住正确的事情，并在正确的时候采取行动。

[Graham Neubig @gneubig]

First CMU Agents class done today w/ @dan_fried, looking forward to getting the video processed and uploaded soon!

[Apodex @Apodex_AI]

Meet Apodex 1.1: Scaling Agentic Intelligence for Complex Work

Open Source Harness: https://t.co/4V9suLx8o5
Open Weights: https://t.co/9CNZ9E5drH

We’re excited to introduce Apodex 1.1, our new model family built to scale agentic intelligence for professional work.

🧠 Frontier-level intelligence for complex work
Apodex 1.1 brings frontier-level agentic performance across complex professional work, scientific research, financial analysis, and deep search.

🤝 Asynchronous Agent Team
Apodex 1.1 can break down complex tasks, coordinate multiple agents in parallel, continuously integrate their findings, and let you step in to guide or redirect the work at any time.

🔬 Open-source research workbench
We’re open-sourcing FrontierAgent—a locally deployable research workbench for the Apodex 1.1 family, including asynchronous Agent Team.

Available now:
🔹 Apodex 1.1 — our most capable frontier model, available through the Apodex online workbench
🔹 Apodex 1.1 mini — open-weight model for running complex work locally
🔹 FrontierAgent — open-source, locally deployable research workbench

🌐 Live workbench: https://t.co/w5l6YI2ZuP
🦾 API platform: https://t.co/kaxuSWvMB4
📃 Paper: https://t.co/CHc4L63U07

[Jacky Kwok @jackyk02]

Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 11x cheaper 💰

As open-source models become more capable, they can now generate large numbers of high-quality candidate solutions and verify their own outputs at very low cost.

For example, we find that sampling just 5 solutions with DeepSeek V4 Flash and ranking them using the same model with LLM-as-a-Verifier can lead to a significant boost in accuracy (79% → 88%), outperforming closed frontier models on Terminal-Bench.

Try it out today: https://t.co/UhudjKDaYI

More on verification scaling in my previous post.

[Jimmy @cyyeh0923]

AI can now generate code much faster than we can understand it.
I think that changes the bottleneck in software engineering.
The problem is no longer:
“How do we write code faster?”
It is increasingly:
“How do we build an accurate mental model of a codebase — and verify that AI-generated changes are actually correct?”

So I built an open-source agent skill called investigate-codebase.

Instead of asking a coding agent to “explain this repo” and getting a confident directory tour, the skill pushes the agent to build the smallest evidence-backed, falsifiable model needed to answer a concrete question.

It supports four investigation modes:
🔍 ORIENT — understand a large or unfamiliar codebase
Trace the important components and produce a focused reading route instead of summarizing every directory.
🧵 TRACE — understand how something actually works
Follow a request, control flow, data flow, state transition, or bug through a thin vertical slice of the system.
💥 IMPACT — understand what a change really changes
Analyze a PR, commit, branch, or diff for behavioral delta, affected contracts, invariants, and blast radius.
✅ VERIFY — check whether generated or changed code is actually correct
Go beyond “tests are green” by looking for independent correctness oracles, boundary cases, invariants, differential checks, runtime evidence, etc.

One principle I care about a lot:
evidence ≠ confidence.
Every important claim is classified as:
SOURCE / RUNTIME / INFERRED / UNKNOWN / CONTRADICTED
So a statically inferred call graph cannot quietly become “this is what happens at runtime,” and an unavailable verification step becomes UNKNOWN rather than a confident guess.

The skill is also deliberately read-only by default.
My broader motivation is that as agents produce larger amounts of code, human understanding and verification throughput may become one of the most important constraints in software engineering.

We need agents that don't just generate more code.
We also need agents that help us compress a codebase into a mental model we can inspect, challenge, and trust.

The skill works with Claude Code, OpenAI Codex, and other hosts that support the skill format.

Open source here:
https://t.co/Cvo2F8ik9v

Would love feedback — especially from people working with large codebases or reviewing a lot of AI-generated code.

#AI #SoftwareEngineering #CodingAgents #LLM #DeveloperTools #OpenSource

[Jimmy @cyyeh0923]

You could also check out a live end-to-end example using DuckDB here: https://t.co/k8v3iEXn6E

[Ahmad @TheAhmadOsman]

Don’t know where to start with Local AI?

Read my Local LLMs From Zero to Hero series

It covers:
- Hardware
- Software
- Models Mechanics
- Everything else necessary

Needs no prior experience

Easy to understand for any background

Local / Opensource AI FTW

[Jeff Dean @JeffDean]

Announcing Discovery Loop!

I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.

♾

Learn more at: https://t.co/Rv3LMdLluK

[山景城小路 @Zen_with_AI]

最会写博客的AI研究员 莉莉安.翁 关于Harness的博客 不得不读！
如果你真的没时间读 不防看看我的总结

《Harness Engineering for Self-Improvement》
作者：Lilian Weng
原文链接：https://t.co/YtD181k3tR

核心观点
文章聚焦递归自我改进（Recursive Self-Improvement, RSI） 的近中期实现路径。作者认为，当前真正可落地的 RSI 不太可能从模型直接改写自己的权重开始，而更可能先从Harness（部署层/编排层） 的工程优化入手。

Harness 的定义：围绕基础模型的外围系统，负责编排执行、思考与规划方式、工具调用与行动、上下文感知与管理、产物存储、结果评估等。它相当于模型与真实世界之间的“运行时 + 操作系统”。
成功的 coding agent（Claude Code、Codex 等）已经证明：Harness 的重要性并不亚于模型本身的原始智能。

三大设计模式
1Workflow Automation（工作流自动化）
目标导向的循环：计划 → 执行 → 观察/测试 → 改进 → 再执行，直到目标达成。模型会分析自身轨迹和失败案例，通过“agent runtime”迭代，而不是静态 prompt。
2File System as Persistent Memory（文件系统作为持久记忆）
长视野任务中，日志、代码 diff、实验记录、错误轨迹等远超上下文窗口。把状态和产物持久化到文件系统（通过 bash 等工具读写），是当前最实用的记忆方案。
3Sub-agent and Backend Jobs（子代理与后台任务）
主代理可并行生成多个子代理，并管理后台任务。关键是把并行过程做成显式、可检查的（日志、状态文件），而不是把所有东西塞进临时对话上下文。
Coding Agent 案例
主流 coding agent 的工具接口已相对稳定，主要包括：
•文件系统操作（glob、grep、read、edit、apply_patch 等）
•Shell 执行
•Git / LSP
•Web 搜索与浏览器
•产物生成
•后台任务与子代理管理（spawn/resume/wait/interrupt agent 等）
Harness 层 vs 核心智能

作者的近中期预测：
1Harness 会朝着元方法论方向进化（优化“如何得到更好答案的机制”，而不只是答案本身）。
2成熟的 Harness 反过来支持自动研究循环，让模型持续改进，同时更强的模型又会防止 Harness 过度工程化。
最终很多 Harness 技巧可能被模型内化（类似 prompt engineering 的演变），但与外部上下文、工具、评估的接口会长期存在。

Harness 优化方法（文章重点）

优化对象从简单到复杂大致演进：prompt → 结构化上下文 → 工作流 → Harness 代码 → 优化器代码本身。
•Context Engineering
◦ACE（Agentic Context Engineering）：把上下文当作不断演化的“playbook”，用 Generator / Reflector / Curator 维护条目化记忆。
◦MCE（Meta Context Engineering）：双层优化——外层进化“如何管理上下文的技能”，内层优化具体上下文内容。
◦Meta-Harness：直接优化“决定如何存取信息”的代码本身。
•Workflow Design
手工设计（AI Scientist、ScientistOne、Autodata）+ 自动搜索（ADAS 用元代理搜索 agent 工作流代码；AFlow 用 MCTS 优化图结构工作流）。
•Self-Improving Harness
把整个 Harness 当作可优化的代码。经典例子是 STOP（Self-Taught Optimizer）：优化“改进器”本身。后续工作进一步强调可观察性和 harness 更新能力 vs 利用能力的分离。
•Evolutionary Search（进化搜索）
AlphaEvolve、Darwin Gödel Machine（DGM）等：维护候选程序/Harness 池，用 LLM 生成修改，按性能筛选。DGM 允许 agent 修改自己的 harness 代码，在 SWE-bench 等任务上取得明显提升。
•Joint Optimization with Model Weights
尝试同时更新 Harness 和模型权重（SIA、Continual Harness 等），但目前证据仍较初步，训练稳定性和 Goodhart 效应是主要挑战。

未来挑战
1评估器太弱（尤其是主观任务如新颖性、品味）。
2长视野任务的记忆与上下文管理。
3对负面结果的忽视与偏差。
4进化过程中多样性崩溃。
5奖励黑客（reward hacking）。
6长期可维护性 vs 短期任务表现。
7人类监督的介入时机与方式。

总体结论
Harness Engineering 是当前通往 RSI 最现实、最可操作的路径之一。它把“如何让模型更好地工作”本身变成可搜索、可优化、甚至可自我改进的对象。代码是通用表示语言，文件系统是实用记忆层，进化搜索和元优化是主要方法论。
近中期更可能看到的是：更强的模型 + 不断自我改进的 Harness 形成正反馈，而不是模型直接重写权重。真正的科学发现级 RSI 仍面临评估、可靠性和长期一致性等根本难题。

[Aarno @TheGlobalMinima]

“harness is the MOAT” absolutely true.

since late 2025, the step up in model capabilities has been diminishing with every release. the real difference is how the system interacting with the model is engineered. the harness decides how the system will turn out, not the model anymore.

the anatomy of a typical harness:
> isolated sandboxes
> file system
> file and coding tools
> state manager
> orchestration runtime

here’s a handbook that covers context engineering for harnesses

[向阳乔木 @vista8]

https://t.co/0oi6uc4fAC

[yibie @yibie]

Matt Pocock 这个 60 秒视频，推荐了一本书，而且他认为在「战略编程」方面，毫无疑问排行第一。

Matt Pocock：#1 战略性编程书籍推荐

"AI 已经吃掉了日常编码和战术性编程。所以很多人问我：怎么学战略性编程？怎么学长期视角？怎么让你能提前看到代码库的问题，让 AI 在你的代码库里有效工作？

我的建议是回到这些老书。今天要说的是 The Pragmatic Programmer。

这本书是我永远推荐的一本。它可能是我读过布局最好的技术书。里面有太多关于'编程时如何战略性思考'的见解。我从中拿了很多东西直接塞进 system prompt 里，效果很好。比如可追溯性（traceability）、'不要超过你的车灯'（don't outrun your headlights）、巧合编程（programming by coincidence）——我们开发者用的很多术语，都源自这本书。

几个月前重读这本书让我确信：真正的软件基础仍然重要。 因为这本书里的每一条，都像是为 AI 时代写的。

So, The Pragmatic Programmer, David Thomas, Andrew Hunt. Get it."

视频：https://t.co/cFZ3z2ohsA
#AI时代 #工程文化 #战略性编程

[Jimmy @cyyeh0923]

https://t.co/ZvYOnC7umF

[Jimmy @cyyeh0923]

I reviewed and reorganized notes from 99 AI Engineering talks into an open-source site.

The talks are grouped into 9 themes:

Agent evals and observability, agent architecture and reliability, context/memory/RAG, security and identity, data infrastructure, training and inference, AI coding, BI/semantic layers, and product strategy.

Each talk includes a short summary, full notes, and the original video.

One recurring lesson: production-grade agents are not mainly a “bigger model” problem.

The hard parts are usually:
— getting the right context
— defining data and semantics clearly
— making workflows observable and debuggable
— recovering or degrading gracefully
— enforcing identity and permission boundaries
— building trustworthy evals

Moving from PoC to production increasingly looks like a systems engineering problem.

The site supports English and Traditional Chinese, plus in-browser highlighting and saved notes.

Feedback and talk recommendations are welcome.

[Xudong Han @Xudong07452910]

最近看到 Anthropic 研究员 Vivek @itsreallyvivek 的一篇长帖，讲怎么真正训练自己的研究能力。

我最有感触的一点是：很多时候，我们以为自己在做研究，其实只是在追热点、刷 Arxiv、看别人讨论什么，然后把那些问题接过来继续做。

这当然也能产出东西，但很容易变成一种科研角色扮演：看起来很忙，输入很多，笔记很多，实验很多，可心里其实并不确定自己到底在追问什么。

Vivek 这篇帖提醒我，研究能力不是某种玄学天赋，而是一套很朴素的习惯：自己选问题，实验前先预测结果；要多读读原paper；把假设、结果和想法变化都写下来，盯着失败案例，而不是只盯着损失曲线。

越往后越觉得，做研究真的很像训练自己。

你给大脑喂什么输入，它就会长出什么 taste；你愿不愿意面对错误，决定了你进步的速度；你写不写下来，决定了你是在诚实迭代，还是说在靠记忆去美化自己。

同质化输入，只会带来同质化输出。

真正拉开差距的可能是谁能在很长时间里，慢慢训练出属于自己的问题感、判断力和纠错系统。

这篇帖很适合迷茫的时候读一遍。

[Artificial Analysis @ArtificialAnlys]

NVIDIA just announced the release of Nemotron 3 Ultra in Jensen Huang's Computex keynote: at 550B parameters (55B active), this is the largest Nemotron 3 model to date, and it is the most intelligent US open weights model

We partnered with @nvidia to evaluate this model for intelligence and speed - these figures use the model’s BF16 weights, but as with Nemotron 3 Super the model will be made available in NVFP4 quantization as well for higher inference performance.

➤ New leader for US open weights intelligence: Nemotron 3 Ultra scores 48 on the Artificial Analysis Intelligence Index. This is well ahead of the next strongest US open weights models, Gemma 4 31B (39), Nemotron 3 Super (36) and gpt-oss-120b (33), but behind the Chinese-led open weights frontier (Kimi K2.6 at 54).

➤ Leading speed for its intelligence: on a pre-release @DeepInfra endpoint, Nemotron 3 Ultra served over 300 tokens per second. Peer models in its size class from China-based labs such as DeepSeek and Moonshot (Kimi) are generally served at speeds of 50-100 tokens per second in the market today. gpt-oss-120b is served at speeds similar to this level, but with significantly lower intelligence.

➤ Largest Nemotron 3 model so far: at approximately 550 billion total parameters and 90% sparsity, Nemotron 3 Ultra is significantly larger than its siblings and is the largest recent US open weights model release

We’ll be sharing additional analysis and full benchmarks at release.

[Jimmy @cyyeh0923]

I’ve always had a deep passion for education.

From working as a tutor when I was a student, to designing learning materials and teaching high school students as part of an academic team in university camps, to now providing consulting services in the AI and data space — these experiences have repeatedly shown me how much I enjoy helping people understand complex ideas and grow through learning.

Today, I’m a software engineer, and I’m ready to bring that same passion for education into how we train and cultivate engineers.

I’m really excited about this.

I’m building something big: an education product for engineers that doesn’t exist in the market yet, designed entirely for the agent era.

Stay tuned. 😊

[Google Gemma @googlegemma]

https://t.co/BvHkG5TaBF