Lahfir

Computer Use | Built agent-desktop, cracked-agent, PILOT, schemalint | 2X founder | 1nder Labs

[Lahfir @mdlahfir]

Codex 的 Computer Use 很棒，但你的 agents 也需要具备电脑操作能力。

这就是 agent-desktop。

灵感来自 @vercel labs 的 agent-browser。

一个基于 Rust 的 CLI，具备桌面自动化能力。

这是唯一一个同时整合了 Mac（正式可用）、Windows 和 Linux 的 CLI。

快照

决策

执行

再次快照

主要的设计难题是上下文大小。

一个朴素的做法是把完整的可访问性树直接丢给模型，但真实应用的树通常非常庞大。Slack 的完整树转储很容易超过 50,000 个 token，这让这种方法变得不切实际。

我最终采用的方法是渐进式骨架遍历：

第一遍：返回一棵浅层树，通常深度为 3，更深的容器会被截断，并标注 children_count

已命名的容器会附带引用，这样 agent 只需请求对应的子树

agent 可以通过 --root @e3 深入到相关区域

这些引用是作用域化的，只有对应子树的引用会失效

在执行操作后，agent 可以只重新查询该区域，而不必对整个应用重新做一次快照

[Lahfir @mdlahfir]

Me saying goodbye to fable yesterday

[Kimi Developers @KimiDevs]

Meet Kimi-K2.7-Code 👀

Here’s what developers should know to fully unlock K2.7-Code potential：

[Lahfir @mdlahfir]

As Siri AI improves over time, apple will open APIs for apps to be able to expose executable tools to Siri that it can take actions in the app. A like_post tool for example that Siri can like the post instead of you manually doing it. So many possibilities.

[Lahfir @mdlahfir]

@dancherrington @JinjingLiang @RhysSullivan Then either agent-browser or agent-desktop should be able to debug it

https://t.co/bePPYOLP4A

[Lahfir @mdlahfir]

@kieranklaassen Done!

https://t.co/FSaKaRkkYt

[Lahfir @mdlahfir]

@trevin https://t.co/bBbtIHjOs3

Been using this and saved around 1.2B tokens overall

[Lahfir @mdlahfir]

@JinjingLiang @RhysSullivan It's an iframe and iframe elements are not interactive (accessibility elements are non accessible inside iframe)

[Lahfir @mdlahfir]

@andrewmccalip @XAwojinrin Won't Stripe Connect make this easier for you?

[Lahfir @mdlahfir]

Now you can do manual rate limits in Codex?

[Lahfir @mdlahfir]

@trevin There's a huge problem that comes with it. Changing a model mid-prompt invalidates the current prompt cache. which is actually catastrophic because it costs more tokens.

[Lahfir @mdlahfir]

IBM quantum computers are $1.6/second of compute

[Lahfir @mdlahfir]

Fable 5

[Lahfir @mdlahfir]

https://t.co/BRiPa7ryR5

[Lahfir @mdlahfir]

Claude has it's own radio station

[Lahfir @mdlahfir]

@trevin @kieranklaassen Amazing. Would you mind sharing your loop flow on the /ce-work /ce-simplify-code and /ce-code-review?

Currently I do

/ce-plan
/ce-work
/ce-simplify-code

LOOP
/ce-code-review -> Fix

/ce-compund and refresh and finally PR

I squeeze in the codex security flow in between as well

[Lahfir @mdlahfir]

@GCrownboy literally

[Lahfir @mdlahfir]

So aftern June 22, it's not on the max plan anymore?

[Lahfir @mdlahfir]

@DanielSmidstrup me reading this after spending 2 years building a product no one is going to use

[Lahfir @mdlahfir]

@TheCryptoCPA Consume extra usage to not show the ads