Ben Kelley

Security product manager at Microsoft. Things I like: software, sports, music theater (I play clarinet, saxophone, flute), PC & tabletop games, sci fi.

[Ben Kelley @benkelley]

@aarondfrancis @samwhoo @DanHollick @laravel @samwhoo massive green flag right here ☝️

Also congrats!!

[Ben Kelley @benkelley]

@sachinrekhi One thing our Design team has tried is sorta the *reverse* flow: automatically grabbing screenshots from the prototype and reconstructing the workflow back in Figma.

So far it has only produced trash 😔 but I'm sure the models will eventually make this viable!

[Ben Kelley @benkelley]

@Bquittem @max_gagliardi +1 to this, I'm reading it to my 5 year old and she's digging it. I just change (or pause to explain) occasional unfamiliar words, and keep sessions pretty short, and let her prance around while I'm reading (not forcing her to sit attentively, which isn't realistic)

[Ben Kelley @benkelley]

@clairevo Fable told me I had a "trilemma."

Yeah it's technically a word, and I knew what it meant from context, but I've literally never seen anyone use that word in my life.

[Ben Kelley @benkelley]

@maiab Omg so it's not just me! I thought maybe my eyes were starting to go (I'm 41, so plausible I guess). I don't have an answer yet, since it doesn't seem to correlate with just 1 room, or bulb age, or brand. Hope others have suggestions we can dig into!

[Orson Scott Card @orsonscottcard]

You don't need advice from editors on rejected manuscripts. 

My short story “Ender's Game” was rejected by Ben Bova at Analog back when that was the top market for a sci-fi story. Ben gave me feedback. He thought the title should be “Professional Soldier” and he said to “cut it in half.”

But I knew he was wrong on both points and submitted it to Jim Baen at Galaxy. He sat on it for a year, and responded to my query with a rejection. There was some kind of explanation, but I don't remember what it was. I concluded at the time that Baen's comments showed that he had barely glanced at the story.

So … I got feedback both times, but it was not helpful. I looked at Ben's rejection again. What was it about the story that made him think it should, let alone COULD, be cut in half?

Apparently it FELT long. What made it feel long? Now, post-Harry Potter, I would call it the quidditch problem. I had too many battles in which the details became tedious. So I cut two battles entirely, merely reporting the outcomes, and shortened another. In retyping the whole manuscript (pre-word-processor, that was the only way to get a clean manuscript), I added new point-of-view material to the point that I had cut only one page in length. So much for “in half.”
But I already knew that my manuscripts did not need cutting — if it wasn't needed, it wouldn't be there in the first place. Even the battles were still there, but instead of showing them, I merely told what happened (so much for the usually asinine advice “show don't tell”), which kept the pace going.

Those changes made, I sent it to Ben again. I did not remind him of what he had advised me to do. I merely told him I liked my title, and said, “I have addressed your other concerns,” which was true. I figured he wouldn't remember what his exact words had been. My answer was a check. That revised story was the basis for my winning the Campbell Award for best new writer.

Did Ben's feedback help? Yes — but his specific advice was not right, and I knew it. On my next two submissions, Ben hated my endings, and I revised as suggested. The fourth submission he rejected outright, and the fifth, and I thought, Am I a one-story writer? I went back to Ender's Game and tried to analyze why it worked. Then, deliberately imitating myself, I wrote “Mikal's Songbird.” Ben bought it, and it received favorable mentions. I was afraid then that I had consigned myself to writing stories about children in jeopardy. But in fact I was writing character stories rather than idea stories. And THAT was how I built a career, not by self-imitation, and not by following editorial suggestions.

I did get wise counsel from David Hartwell on my novel Wyrms, but that was on a book that was already under contract, and it was story feedback, not style. I got wise counsel from Beth Meacham, too, on various books over the years — but again, only on books that were under contract. I also received appallingly stupid advice from the editor of my novel Saints, which temporarily destroyed the book's marketability; after that, I was allowed to go back to my original structure and save the book — now it's one of my best.

Editors don't know more than you about your story. They especially don't know why they decide to accept or reject stories. YOU have to know what your story needs to be, and take only advice that you believe in.

Your best counselor on a story nobody bought is TIME. Let some time pass and then reread the story. Don't even think about why it Didn't Work. Instead, think about what DOES work, and then write it again, a complete rewrite, keeping nothing from the previous draft. Find the right protagonist and begin at the beginning — the point where the protagonist first gets involved with the events of the story. Be inventive — the failed first draft no longer exists, so you're not bound by any of your earlier decisions. THAT is how you resurrect a good idea you did not succeed with on your first try.

[Aakash Gupta @aakashgupta]

Anthropic has 454 open roles. The company is hiring software engineers at $320K-$405K. Their CEO, Dario, said three months ago that coding is "going away first, then all of software engineering."

The paradox resolves instantly.

Dario's engineers told him they don't write code anymore. They let Claude write it. They edit. They review. They architect. They didn't lose their jobs. They got faster. Anthropic grew from a small research lab to 1,500 employees in four years, adding engineers the entire time.

This has played out five times in computing history. Compilers replaced assembly. Frameworks replaced boilerplate. Cloud replaced server management. Every prediction was the same: most programmers won't be needed. Every result was the same: the number of engineers grew.

The global software engineer pool went from roughly 5 million in 2010 to 28.7 million today. BLS projects 17% growth in US software developer roles through 2033, adding 304,000 positions. The pool is projected to hit 45 million by 2030.

When building software gets cheaper, more problems become worth solving with software. A startup that needed 10 engineers now needs 3. But 50 companies that couldn't afford to build at all now can. The denominator shrinks. The numerator explodes.

Meta's engineering headcount is up 19% from January 2022. Google's is up 16%. Apple, 13%. These companies adopted AI coding tools years ago. They're using Copilot and Claude Code daily. They're hiring more engineers than before those tools existed.

Every generation of "coding is dead" content creates two cohorts: engineers who freeze up, and engineers who build 10x more with the new tools. The second group has won every single time.

[Michael @michael_chomsky]

Garry is kinda correct here, but is oversimplifying memory. Harrison (the author of the original article) makes a very good point but also makes memory sound easier than it is.

(before reading this article, note that I wrote down my thoughts and then passed it through Claude Code. I read every word. read it like like a coworker's Claude Code output)

Let me start with where Garry is right, because he IS right about something important.

Git-backed markdown is a memory format that is simultaneously human-readable, version-controlled, diffable, and greppable. No database gives you all four at once by default. If your agent's memory is an opaque blob in someone else's database, you have no idea what it "knows" about you. You can't correct it, can't diff it, can't even look at it.

That matters. A lot. I agree with this completely, and it's the right starting posture.

But it's a storage format. It's not a memory system. And the difference matters more than most people in this debate seem to realize.

Harrison's argument is different. He says memory is tied to the harness, the harness must be open, therefore you should use their open harness. The first two points are correct, the third is iffy because it assumes you want to be responsible for memory, which is hard (probably right but not a trivial decision). But that core insight--that the harness and memory can't be separated--is real and more important than people give it credit for.

Let me explain why, and then why everyone in this debate is still underselling the difficulty.

The harness owns the critical moment

The most important time for memory to be created or updated is during compaction.

Compaction is when the context window fills up and the agent compresses everything into a summary. Information that doesn't survive the summary is gone--not archived, gone. This is memory triage, and the harness controls it. Always.

OpenCode, OpenClaw, and Hermes all handle this. OpenClaw does it by default. OpenCode's SDK exposes compaction hooks--you can listen for session.compacting events and handle memory yourself. This is a great place for memory logic to live.

Now look at what Codex does: it produces an opaque, encrypted compaction summary that isn't usable outside the OpenAI ecosystem. Harrison himself flagged this in his article. This isn't just vendor lock-in, it's architectural lock-in by design.

Harrison is right to be alarmed by this. Garry is right that being "above the API line" matters. But neither one grapples with what actually makes memory hard once you've decided to own it.

Where files break down: forgetting

Garry's model inherits all the strengths of git: version history, diffs, blame, rollback.

But git's greatest strength is the core problem: nothing is ever truly forgotten.

When do you choose to forget a memory? How do you know it's outdated? You changed jobs six months ago--is the memory about your old team's coding standards still valid? Your codebase migrated from REST to GraphQL--are the API pattern memories stale, or still useful for legacy endpoints that still exist?

With files, you can delete them. But you need to know they exist AND that they're stale. And you need to check this proactively, because nobody is going to tell you.

This is actually a structured problem with real solutions starting to emerge. Zep's Graphiti engine uses what they call bi-temporal knowledge graphs--every fact gets timestamps for when the system recorded it AND when it was true in the real world. Facts are invalidated, not deleted. You can query "what did I know about X on March 15th" separately from "what is currently true about X."

Most memory providers are converging on some version of this. Supermemory has a graph-based system. Hydra is moving toward mixed graph/vector approaches. Mem0 added graph memory. This convergence is telling--it means the industry is collectively figuring out that flat files and pure vector search aren't enough for temporal reasoning.

Files don't have temporal validity windows. Git has history, but history and validity are different things. Knowing a file changed on March 15th doesn't tell you whether its contents are still true today.

Then there's the injection problem.

OpenClaw's memory.md is a trivial file with memories, injected into context every time, updated at compaction.

It's also fully observable because it's just.. a file. This was a genuine innovation and a really good idea.

But my OpenClaw installation clients keep running into the same wall: not all memory needs to be in context every time, and there's a ceiling on how much fits. Claude Code caps MEMORY.md at something like 200 lines. After that, the content just doesn't get loaded at session start. You lose it.

Most memory systems solve this with a reactive search_memories tool. The agent needs something, searches for it, finds it. Fine. But what happens when the agent doesn't know it should be searching?

A coding agent drifts off-track and violates a pattern your team agreed on three months ago. The memory exists. The agent didn't search for it because it didn't know it was relevant. There was no trigger. It just.. didn't know what it didn't know.

This is the proactive injection problem, and it's the hardest open question in memory right now.

There IS real research on this. MemGuide ranks candidate memories by something they call "marginal slot-completion gain"--basically asking "would injecting this memory fill a gap the agent actually needs right now?" PRIME takes a different angle, building proactive reasoning through iterative memory evolution. These are promising but none of them are production-ready for synchronous agents where you can't afford an extra inference round on every turn.

Mesa's Saguaro is interesting here. After every agent turn, it spawns a separate LLM that reviews what the agent just did against the full codebase. If the agent is drifting, it corrects course. They kinda built a memory system without calling it one. It's just really slow because you're doing LLM inference after every single turn.

Supermemory proved the logical extreme of this in their April Fools experiment: throw enough inference at the problem (eight parallel prompt variants, a dozen model calls per query) and you beat basically every memory benchmark. 98.6% accuracy. But the per-query cost is absurd. Their actual production system--the graph-based one--scores lower on benchmarks but is, you know, usable. For async agents where latency doesn't matter, brute force actually make sense. Not for Jarvis, my OpenClaw agent.

Where files break down: relationships and search

If you store everything as files, there's no way to search "all people I know" or "bugs I often make in this codebase" unless the agent happens to organize memories that way. And it won't, because agents are inconsistent organizers.

Concepts and relationships aren't flat. They're graphs. A person connects to a company, a project, a set of conversations. A coding pattern connects to a language, a framework, a set of past mistakes. Files can represent individual nodes but they can't represent the edges without becoming something else entirely.

So you solve it by adding structured search over your markdown files. Oops, you've built a database!

https://t.co/ICR6XoxJyn

@swyx wrote about this years ago: developers who avoid using a real database inevitably build one, badly, through incremental decisions. You start with files, add search, add indexing, add schemas, add conflict resolution, and suddenly you have Postgres except worse.

This is actually what happened with GBrain--Garry's own implementation of "memory is markdown, brain is a git repo." The files go in as markdown. But underneath? Postgres and pgvector for hybrid search. The markdown is the interface, the database is the engine. Even the strongest advocate for file-based memory needed a database to make it actually work.

The Composio model

Here's something I think is underexplored: portable memory across agents.

The same way Composio lets you move integrations across agents, some memory providers are moving toward letting you own and share memories across Claude, ChatGPT, OpenClaw, whatever. Your memories live in a vault you control, and each agent reads from and writes to it.

I'd call this the Composio model of memory. It's a good idea and more providers should pursue it.

But then you're potentially running two memory systems--one inside the harness (memory.md, CLAUDE.md, whatever the harness does at compaction) and one external. What a mess.

Hermes and OpenClaw both let the user choose their memory backend. Flexibility sounds great until you realize it means the system has to handle the possibility that memory is in two places at once, managed by two different things, with two different update cadences. I still think giving users this choice is the right call. But it is genuinely complicated.

The cost that nobody talks about

Every sophisticated memory system costs inference tokens. Letta's self-editing model--where the agent actively decides what to remember during reasoning via tool calls--is the most architecturally interesting approach I've seen. The agent curates its own memory as a first-class part of thinking. But every core_memory_replace call is tokens. Mesa's per-turn review is a whole extra LLM call. Supermemory's brute force approach is a dozen.

File-based memory is effectively free. Read a file, inject it, done.

The bar for beating memory.md isn't just "is it smarter?" It's "is it enough smarter to justify the cost?" And for most use cases today, the honest answer is no.

But here's something that should make people pay attention: recent benchmarks on agentic memory (AMA-Bench among others) are finding that the design of your memory system matters way more than which model you're running. We're talking maybe an order of magnitude more variance from architecture choices than from model scaling. The architecture matters enormously. It just also costs real money, and that tension is why most production systems still use the simple thing.

The unsolved problems

Recent research has started to identify what a memory system actually needs to do well:

Accurate retrieval--find the right memory when asked.
Learning in real time--update what you know from new information as it comes in.
Long-range understanding--connect things across sessions that happened weeks apart.
Selective forgetting--know when a memory is stale and stop using it.

No current system is good at all four. Graph-based systems handle forgetting and long-range connections better than anything else, which is probably why everyone is converging on them. Letta does well on retrieval and real-time learning. File-based systems do ok on retrieval and struggle with the rest.

Now add multi-agent coordination. Multiple agents on the same filesystem. Multiple people cooperating with agents on different projects. Who organizes the memory? Who resolves conflicts? Do we deploy an async agent to consolidate memories at compaction time? At session end? On a cron job overnight?

How do we prioritize recent memories over old ones? How much control should the agent have over its own memory? How do we handle that some people want aggressive memory and some want minimal? And they might want to export it and bring it to another agent!

These aren't rhetorical questions I'm asking to sound smart. I deal with these every week deploying agents for clients. Nobody has good answers.

Benchmarks exist now. They're just not reliable.

A year ago there were no memory benchmarks worth talking about. That's changed. LOCOMO, LongMemEval, AMA-Bench, MemoryAgentBench all exist. There's even an ICLR workshop this year dedicated to agent memory.

But here's the problem: evaluation choices that look like implementation details--the prompt you use for the judge model, the scoring methodology, the answer generation setup--can swing accuracy by double digits. Supermemory showed this directly when they demonstrated you could score 98.6% by letting any of eight prompt variants count as correct. That's not a benchmark result. That's a configuration choice dressed up as one.

So we have benchmarks. They're just not trustworthy enough to settle any debates. If you overcomplicate your memory system, you still can't be sure it's actually outperforming a memory.md other than vibes. Just vibes with numbers attached.

Nobody has memory right

Not Garry, not Harrison, not OpenClaw, not Letta, not Zep, not Supermemory, not Mem0. Nobody.

Garry's instinct--keep it simple, keep it readable, keep it yours--is the right starting posture. Harrison's instinct--the harness and memory are inseparable, own both of them--is architecturally correct. Sarah Wooders' framing--memory is context management, not a retrieval problem--is the most precise explanation of why this is so hard.

But memory.md is not the end state. It's the beginning.

It's the simplest thing that works, and for most use cases today it's the right choice. Not because it's good. Because everything else is either too expensive, too complex, too slow, or too unproven to justify the leap.

The gap will close. The research is real, the providers are converging on graphs, and the benchmarks are slowly forming.

But if anyone tells you they've solved memory, they haven't. They've solved one of the four problems and they're hoping you don't ask about the other three.

[Marc Andreessen 🇺🇸 @pmarca]

Claude knows! —>

The Lump of Labor Fallacy and Why AGI Unemployment Panic Is Economically Illiterate
Let me lay this out with full rigor, because this argument deserves to be prosecuted completely rather than waved away with a sound bite.

I. What the Lump of Labor Fallacy Actually Is
The lump of labor fallacy is the assumption that there exists a fixed, finite quantity of work in an economy — a lump — such that if a machine (or an immigrant, or a woman entering the workforce) does some of it, there is necessarily less left for human workers to do. It treats employment as a zero-sum pie.
The fallacy was named and formalized in the early 20th century but the error it describes is far older. It animated the Luddite riots of 1811–1816, where English textile workers destroyed power looms convinced that the machines would steal their jobs permanently. It drove opposition to the spinning jenny, the cotton gin, the mechanical reaper, the steam engine, the telegraph, the railroad, the automobile assembly line, the personal computer, and every other major labor-displacing technology in the history of industrial civilization.
Every single time, the catastrophists were wrong. Not partially wrong. Structurally, fundamentally, categorically wrong — because they misunderstood the nature of economic production itself.
The reason the fixed-pie assumption fails is this: demand is not fixed. Work generates income. Income generates demand for goods and services. Demand for goods and services generates new categories of work. This is an engine, not a reservoir. When you drain some of the reservoir with a machine, the engine speeds up and refills it — and often refills it past its previous level.

II. The Classical Economic Mechanism That Destroys the Fallacy
To understand why the lump-of-labor assumption is wrong about AGI, you need to understand the precise mechanism by which technological unemployment resolves itself. There are four distinct channels, all operating simultaneously:
Channel 1: The Productivity-Demand Feedback Loop (Say’s Law, Modified)
When a technology increases the productivity of labor or replaces labor entirely in a given task, it lowers the cost of producing whatever that task was part of. Lower production costs mean either:
∙Lower prices for consumers (real purchasing power rises), or
∙Higher profits for producers (which get reinvested, distributed as dividends, or spent as wages for other workers), or
∙Both.
Either way, aggregate real income in the economy rises. That additional real income does not evaporate. It gets spent on something — including goods and services that didn’t previously exist or were previously too expensive to consume at scale. That spending creates demand. That demand creates jobs.
This is not a theoretical conjecture. The average American in 1900 spent roughly 43% of their income on food. Today it’s around 10%. Agricultural mechanization didn’t produce a nation of starving unemployed farm laborers — it freed up 33% of household income to be spent on automobiles, television sets, air conditioning, healthcare, education, travel, smartphones, and streaming services, most of which didn’t exist as industries in 1900. The workers who left farms went to factories, then to offices, then to service industries, then to information industries. The economy didn’t run out of work. It metamorphosed.

[Sam Rose @samwhoo]

StackOverflow graph of questions asked per month.

Holy shit.

[Ben Kelley @benkelley]

@SickOfThis50547 @JourdanJD My kid (4) still says "Can we go to Old McDonald's?" and I'm physically incapable of correcting her

[Ben Kelley @benkelley]

@clairevo Thanks for working on this, since I'm uhh... NOT succinct 😅 so I really value ChatPRD's help on that

[Ben Kelley @benkelley]

@petergyang @clairevo Totally see your point, but I'm not mad at this choice, because A) I only want that next step maybe half the time (if i just want a quick answer, or it guesses wrong), and B) I totally get they want to trim costs where possible. Like the Netflix "Are you still there?" message.

[Ben Kelley @benkelley]

@clairevo Yep, this is why I hate Slack messages that are just "Hi Ben."

[Ben Kelley @benkelley]

@sm Dang, what the heck did I miss today? Nutella and electric cars and now alcohol? I pay attention at work for ONE day... 😅

[Ben Kelley @benkelley]

I feel personally attacked 😅

But seriously, it's amazing how often we just need a nap or a juice box. We're fundamentally still the same people as when we were 3 years old, just with more layers built on top.

[Kevin Yien @kevinyien]

@jmduke is straight up peering into the soul of every productivity nerd with this one

[Ben Kelley @benkelley]

@DanielleMorrill Yeah, for an ordinary PC setup (single GPU), RTX 4090 is probably your best bet... unless you want to hunt down a used A100, or an AMD MI200/300. But $$$$ 😅

Also, if you don't mind waiting until January-ish, the RTX 5080 and 5090 are coming soon!

[Ben Kelley @benkelley]

@LukeYoungblood @brian_armstrong @coinbase Heh yep! I entered this thread to make that same reference 😆

[Ben Kelley @benkelley]

@Austen @jeff_weinstein +1, came here to say this