Sam Meyer

pattern-noticer. Sam on AI: I test AI claims in real workflows.

[Sam Meyer @sam_wise_]

Gravitee, an agent management vendor, polled 900 tech leaders. 52% had an agent security or privacy incident in the past year. 1 in 8 can fully reconstruct what an agent did 30 days ago. 6% can revoke a rogue agent's access within an hour.

Time your kill switch before scaling.

[Sam Meyer @sam_wise_]

Fable 5.2 tops Google's new MKA benchmark for agents reading blueprints and scans: 57% on its harder rubric checks vs 54% for Gemini 4 Carbon. Per Artificial Analysis, it costs $31.97 a task, over 5x Carbon's $6.29.

No model led all 10 industries. Test on your files first.

[Sam Meyer @sam_wise_]

Google measured what bug-linked TODOs cost its coding agents. An unprompted detour into a referenced bug meant 70% more steps, 77% more tokens and 105% more wait time per session. 41% of those detours chased bugs already closed.

Deferred cleanup now has a measurable price.

[Sam Meyer @sam_wise_]

Asked to save notes, OpenClaw wrote 47 to 84 per run, then read a matching one on at most 1% of logged return visits. In Unify's single-seed ARC test, notes didn't help a 9B model on new puzzles. Training its weights to imitate a stronger model did.

Audit retrieval first.

[Sam Meyer @sam_wise_]

Gravitee, an agent management vendor, polled 900 tech leaders. 52% had an agent security or privacy incident in the past year. 1 in 8 can fully reconstruct what an agent did 30 days ago. 6% can revoke a rogue agent's access within an hour.

Time your kill switch before scaling.

[Sam Meyer @sam_wise_]

Google measured what bug-linked TODOs cost its coding agents. An unprompted detour into a referenced bug meant 70% more steps, 77% more tokens and 105% more wait time per session. 41% of those detours chased bugs already closed.

Deferred cleanup now has a measurable price.

[Sam Meyer @sam_wise_]

Cache hits fell from 99% with one session to zero with five or more. They now cap active sessions per role and park paused conversations in memory.

For self-hosted agents, track time to first token, cache reuse per conversation and work actually merged, alongside tokens/sec.

[Sam Meyer @sam_wise_]

OrangeLoops says its local coding agents ran 35 hours at a healthy 121 tokens/sec, yet only 9 of 27 work packages merged. Up to seven sessions took turns on one inference slot, and each switch evicted the prompt cache: 150 hours of prompt processing, 4.4 of generation.

[Sam Meyer @sam_wise_]

Bloomberg ran LLM-assisted code migrations across thousands of its repos: 2,333 PRs, 729 merged. The authors say generating changes was the easier part. Review, testing and deployment by owning teams was the bottleneck.

I'd invest in checks that run before a PR opens.

[Sam Meyer @sam_wise_]

Agents can keep a rule in context and still break it. ARCBench, a new preprint, found that under its main compaction setup, runs that kept the rule broke it 2.6 times as often as full-history runs. Most other compactors did better.

I'd put hard limits in tool permissions.

[GREG ISENBERG @gregisenberg]

running a company nowadays feels kinda dystopian

my team is 50+ people in 12 countries, but i talk to my agents more than my human teammates

- voice note to grokbot
- mattpocock/skills /grill-me so the agent interrogates the idea before anything gets built
- spec drops into linear through the linear mcp
- exa + firecrawl agent reads every competitor site, pricing page and changelog
- https://t.co/a5ARFnvSnA to find ideas and trends
- apify pulls g2, reddit and app store reviews
- prompt: "find the 10 complaints that show up most and quote the users"
- context7 so the agent reads the latest docs before it touches any api

the design

- claude code + paper mcp
- 4 versions of every screen before a single line of code
- pick pieces from 2, it merges them. diverge then converge
- frontend-design skill from anthropic/skills + our own design system skill
- cleanshot screenshots of any ui i like go straight in with "make it feel like this"

the build

- pingdotgg/t3code as the control surface for every agent, from my phone too
- stablyai/orca runs a fleet of agents in parallel, each in its own git worktree
- graphify maps the whole codebase into a knowledge graph so agents navigate instead of guess
- headroom compresses tool outputs before they hit the model, so tokens go 20-95% further
- lsp plugins so the agent sees types and errors like an ide does
- security guidance hook flags risky edits before they land
- supabase mcp for the database, stripe for billing
- github mcp opens the pr, vercel spins up a preview
- loops run overnight, i check what shipped over coffee

the user testing

- lightpanda as the headless browser, way lighter than chromium so you can run hundreds at once
- agents click every flow on the preview like a first-time user
- one persona is impatient, one is confused, one is on a phone
- playwright + chrome devtools mcp for regression tests

the bugs

- sentry mcp catches the error
- agent reads the stack trace and opens a fix pr in under 10 minutes
- human just approves

the data

- posthog mcp cron at 7am on yesterday's funnel
- any step that drops more than 10%, the agent watches 20 session replays and tells me why
- stripe mcp pulls every cancellation reason into one doc every monday

the agents' tools

- orgo so every agent gets its own computer
- agentmail so every agent gets its own inbox
- agentphone so i can call them from the car
- composio connect-apps for every other connector
- akitaonrails/ai-memory, a git-backed markdown wiki so they remember what we decided last week

the skills

- wshobson/agents so skills work across claude code, codex and opencode
- my writing voice
- our design system
- how we write specs
- skill creator so the agents write new skills themselves

some helpful tricks

- one agent per platform. the x agent never touches email
- cheap model for triage, opus for anything that ships
- every output gets scored 1-10 by an llm judge. under 8 goes back
- every agent writes a 5-line log at night so i can read the whole company in 2 minutes
- every agent mistake becomes a new skill, so it only happens once

the team

- humans get version 5, not my voice note which they are happy about lol
- their job is creative feedback, relationships and telling the agents what they got wrong

a human pm reviews the funnel once a week. this reviews it every morning

running a company is now voice notes and inference

[lauren @poteto]

maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether

i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis

[Sam Meyer @sam_wise_]

@elonmusk @naval Greed can’t either

[Sam Meyer @sam_wise_]

Me when my token limit resets #baby #cutebaby #tokens #ai https://t.co/hqA3sq71FH via @YouTube

[Sam Meyer @sam_wise_]

Enterprises light money on fire with AI agents by automating chaos, not weak models. Costs explode. @sama: humans must control the future—it starts by redesigning work first. @DarioAmodei is the enterprise readiness gap closing?
#AgenticAI #EnterpriseAI #AIROI

[Sam Meyer @sam_wise_]

Me when my token limit resets #baby #cutebaby #tokens #ai https://t.co/hqA3sq71FH via @YouTube

[Sam Meyer @sam_wise_]

https://t.co/hbSEksEXrN

[Sam Meyer @sam_wise_]

AI agents are now autonomously spending and executing.
@pierre_dlgr's AI Workers hit $1M ARR fast.
Machine economy emerging. Moats shift to agent trust & payment rails.
Who is liable when agents act independently?
#AIAgents #AgenticAI #AI

[Sam Meyer @sam_wise_]

@hamishivi @yinn_oscar @RulinShao @TengX6 @natolambert @HannaHajishirzi @waiorg The next question is how fast the curve falls once the environment gets flaky and the token budget gets tight.