"Not context dumping, I’m careful about what goes into https://t.co/nR57q6ZX4M and agents. Too much noise and Claude gets confused. Too little and it defaults to generic advice. The goal is precision: enough information to eliminate guessing"
https://t.co/CJAqdD2Fwo
I've got an agent in a loop optimizing a renderer with the goal to minimize frame times (and tests to measure). It got times down from 88ms to 2ms and allocations down from ~150K to 500. Sounds good, right? Wrong. This is exactly why agent psychosis is a big fucking problem.
As an experiment, I rewrote the Ghostty core render state in Go, with access to identically laid out data structures as Ghostty and the exact same validation tests. I made a purposely naive renderer (simple, correct, but slow). 88ms per frame with 150,000 allocations (horrendous, lol)!
I then kickstarted a Ralph loop to bring the frame times down. I told it it can't modify input data structures or the public API or tests (they're correct), but it can do anything else it wants. It got to work.
It has worked for about 4 hours. I've spent around $350 on this experiment so far. The results?
88ms => 1.5ms
150K allocs => ~500 allocs
Incredible right? Nope.
My hand-written renderer I ported has frame times (same benchmark) of ~20us (0.020ms) and 0 allocations in the update path.
This is the problem with psychosis and lacking systems understanding. If you don't understand the system, you're going to accept that this is an incredible result. If you understand the system, you'll see better solutions immediately and can do roughly 75x better on throughput.
The people who blindly trust agent output are in the former camp. They're sheeple, overdrinking from a fountain of mediocrity.
Standard disclaimer: I use AI all the time. I like AI. The point I'm making is to not blindly accept results. Think. Analyze. Learn.
Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.
@mitchellh the answer is always something in the middle. and you need to engineering judgement to decide what to optimize for when and where. i feel like you think that judgement is highly titled towards MTTR while it shouldn’t be.
@jarredsumner i think all of this still can be true but still have a human steering the agent and the level of steering can be different. @mitchellh vouch system feels like a good take to deal with this for next X years.
@bcherny does batch skill work only for single repo? what about migrate across all repositories which could be 50+
I recently did migrations via cloning all in single folder and then claude act as orchestrator and spawn agent teams with a specific migration skill file. worked well
@EthanLipnik 👋 Early versions of Claude Code used RAG + a local vector db, but we found pretty quickly that agentic search generally works better. It is also simpler and doesn’t have the same issues around security, privacy, staleness, and reliability.
Okay, I'm in a bit of a sentimental mood (it's those Sunday evenings, man), but here's one thing that I keep thinking about:
I wish we all would respond to the loss that some programmers feel right now with a bit more grace.
Some reactions can come across as "it's always been about shipping, dummy" or "huh-duh it's never been about the code, always the results".
And that feels a bit deaf, doesn't it? I mean, let's be honest, for many of us it's been about more than that.
Lots of programmers — myself included! — feel a great amount of joy when they see (or even better: write) the perfect line of code, where abstraction and syntax and semantics meet and hug and everything comes together.
We travelled to meetups and conferences and had hour long conversations with strangers about favorite styles of writing code, about keyboards, colorschemes, keybindings, syntactic flourishes, ways to comment code, cherished parts of standard libs, beautiful ways to rewrite a function that we've all written.
I know a guy who said he makes life decisions based on whether he gets to work with a certain programming language more or less.
Digesting the fact that things are changing and that some things we cherished might be replaced by other things — that takes a different amount of time for different people.
Linters have become an essential part of my workflow. Biome turned all the way up: all errors, all nursery features. golanglint-ci ... same thing. Maximum linting. Don't let your agent code without it.
So I made one for zig too.
https://t.co/5wBMQTqfRf
@threepointone IMO we all need to get much, much better at reading and digesting vast amounts of code, now that writing it has become automated and a commodity.
Code comprehension was always an important skill; but even more so now. Looking at large diffs and being able to zero in on specifics
@copyconstruct totally agree on this part. but understanding when shipping what the code is suppose todo yes. 3 months later? i would have to understand it again anyway. doesn’t matter if AI wrote it or me 😂