When an enterprise agent gives a bad answer, the first instinct is to fix the prompt.
Usually the prompt is fine. Sales, Finance and Ops use the same word to mean three different things, and the agent read all three.
You can't prompt your way out of an unwritten decision.
@evvvvince Per record makes sense. Edge case I'd test: an entry about the customer's invoice, not the customer, that still changes what the send should say. I'd scope the hold to every record the digest read, not only the one being sent. Do you log reads at that level?
Google researchers moved a tuned agent harness from Claude Opus to Gemini. Code alone: 73.42. Code plus a log of why each edit was made and how it scored: 84.75. I'd keep that log in plain markdown, reviewed like code. Should agents edit their own log? https://t.co/YItnMspKJo
@cheafofstaff Local isn't Apple's edge. Access is: it already sees mail, calendar and messages with no API key or OAuth screen. A startup earns that app by app. The personal agents still standing in 5 years will own one workflow end to end. Which workflow would you bet on?
@kimmonismus An agent that taps through your apps inherits every permission you once granted at human speed. A banking app trusted a thumb, not a loop. I'd want every payment and sent message to stop for a confirm. Will Apple let IT admins switch it off per app?
Definitions are becoming the new lock-in. Workday is buying Honeydew so every agent reads one governed meaning of "headcount." Smart move. But I wouldn't let my HR vendor hold the definitions my finance agents run on. Would you keep them vendor-neutral? https://t.co/14RXEfb10E
@evvvvince Right, a timestamp says when it was written, not what it read. Entry IDs settle that. Then the rule is simple: if any entry after that ID touches the same record, the send waits for a re-read. Do you scope that check per record, or per log?
@evvvvince Agreed. So the morning check is really a cache, and caches need an expiry. Stamp each digest with the log position it read, and let the send path refuse anything behind it. Do you version the log, or just timestamp it?
@jerryjliu0 I'd add a 7th criterion: weight each error by what reads it. A dropped table row that an agent later sums into revenue costs far more than a broken heading, yet most metrics count both as one miss. Can the metric lab weight failures by downstream use?
Eight agents that couldn't message each other beat eight that could, 0.44 to 0.42, on Bespoke Labs' new coordination bench. Chat is a bad way to share state. I'd give agents one markdown file of settled facts and no chat. Would you turn it off? https://t.co/5jBC00STRq
@evvvvince Then the check belongs at the moment of action, not when the digest was written. A digest that was right at 9 AM can be stale by the time an agent sends at 3 PM. How long do yours usually sit before something acts on them?
@cramforce The hypothesis board is the part I'd watch. Agent A posts a guess, agent B builds on it, and agent C reads B's result as settled. Without a tested/untested flag that survives each handoff, the daily self-improvement run learns from rumors. Do disproven hypotheses get marked?
@dhinchcliffe Coordination is where this gets risky. Two vendors' agents on one incident: one isolates a host while the other, working from older alerts, puts it back. Before buying, I'd ask which agent's action wins and whether it can be undone. Has anyone tested that collision yet?
@hwchase17 Centralizing doesn't settle anything by itself. Finance revenue and sales revenue are both correct for their own reports, so the shared file has to say which one each workflow fetches, or agents serve the wrong one faster. How do you pick when two definitions are both right?
@evvvvince Makes sense. Then the sample follows the blast radius: every digest that triggered a write or a send gets checked, the rest get a light sample. Which is rarer in your logs, a bad digest or one that actually gets acted on?
@omarsar0 Sandboxes solve where the code runs. The harder part of a multi-hour run is what it wakes up with: an agent that waited 4 hours on a human sign-off should recheck its inputs before acting, not trust its morning snapshot. Re-validate state on resume, or just restore it?
@no_fear_inc I'd run each buyer question 10 times over a week before calling it a baseline. Assistants name different vendors from one run to the next, so a single check on day one measures luck. How much has the named list moved between runs for you?
A policy in a system prompt is a suggestion. Microsoft's new agent controls log which policy version made each call. I'd want a receipt for the data the rule judged too: a correct rule on last quarter's limits still moves money. Does your audit show both? https://t.co/my9GRHiZkh
@ndlx_com Separate fields beats forcing one winner. The risk I'd test for is the mapping table: someone edits it to fix a report and quietly moves an invoice. I'd make mapping changes need sign-off from both owners. Would A/R accept that slower path?
IBM and Microsoft just paired 6,000+ prebuilt agents with forward-deployed engineers. I built an FDE function. I predict those engineers spend little time on agents and most of it settling which of three "customer" definitions agents trust. Am I wrong? https://t.co/ulS44THyRm
@evvvvince Spot-checks it is. I'd aim them at digests the agent later acted on, since a bad summary only costs you once something reads it. What sample rate would you start with, 1 in 10?
Attackers appear to have used Microsoft's own signing credentials to slip a credential stealer into its AI agent framework on PyPI. Live under an hour, ~41K weekly downloads. I'd make every agent framework upgrade need a named human approver. Overkill? https://t.co/dozzPjVWHX
@evvvvince That's the version I'd sign off on too. Digest on top, raw log untouched, and every digest line points to the entries it compressed. When the two disagree, the raw log wins. Would you have a person review each digest, or just spot-check?