Matt Roy

I own a computer. Notes on agent harnesses, evals, tools, and infra.

[Socket @SocketSecurity]

🚨 LiteLLM, a Python package for connecting to AI model providers with 92M+ monthly downloads, has been compromised.

Version 1.107.0 ships credential-stealing malware through PyPI wheels and Docker images, targeting cloud credentials, developer secrets, and AI tools.

[Matt Roy @themattroy]

@LeoBuilds_ If you clearly define all your requirements and give it a way to verify its own work, it gets anything you throw at it done.

[Matt Roy @themattroy]

@housecor It’s 500 because no one is actually reading it. I’m sure there are conflicting instructions in there. Have a model audit the whole thing and you’ll definitely find a few that can be deleted.

[Matt Roy @themattroy]

This is long overdue. Finally an update to my favourite lite model. A lot to do tonight.

[Claude @claudeai]

Introducing Claude Haiku 5.5: the cheapest, fastest, and most capable small model we’ve ever released.

On average, it costs around 75% less to run than Claude Haiku 4.5.

[Claude @claudeai]

Introducing Claude Haiku 5.5: the cheapest, fastest, and most capable small model we’ve ever released.

On average, it costs around 75% less to run than Claude Haiku 4.5.

[james hawkins @james406]

engineers don't even explain their code anymore they just look at you like this

[Matt Roy @themattroy]

@dhh I think people are missing the point here. Say it’s a knock off for now, if maintained and development continues how long before it can rival the actual thing.

[Philipp Keller @philkellr]

Do we actually have better software since AI?

What company, what product got 2x better in the past 3 years?

Vibecoding feels faster but the production gain is 20% at most

[Matt Roy @themattroy]

@bcherny @reasonableglyph @PerceptualPeak Instead of outright blocking, can it prompt for permission?

[luffy @0xluffy]

Introducing Guided Reviews: the better way to review a PR

A big diff is shown in file order, so you spend the first half hour jumping between files, working out what the PR does

Guided Reviews does that for you

- Overview shows the before vs after like a show-me. This way you understand what's changed on a high level
- Files are grouped into chapters, each explained in plain words so you review one idea at a time
- Only pay attention to what matters! The core change comes first. Database, tests and generated files come last

[Mo @atmoio]

The Markdown thing is getting out of hand

[Matt Roy @themattroy]

@philkellr I just do something else or test whatever has already been done, then come up with new ideas or approaches to just to feel involved.

[Ben Holmes @BHolmesDev]

Starting to think about engineering work as "product engineering tasks" and "factory engineering tasks."

Product engineering: improve the product
Factory engineering: improve the agents that improve the product

I heard @poteto call this "gardening," which I really like!

I think of these categories as inner and outer loops, where the inner loop is your team of agents building everything, and the outer loop is your team of supervisors / gardeners watching over those agents and spotting room for improvement.

We're exploring how to automate that second loop. Right now, we have one "outer loop" of agents on our software factory that focuses on agent skill improvement.

One batch of agents scores the quality of agent conversations:
- was the agent efficient, or did it waste tool calls trying to understand its environment?
- did the agent write good code by our standards, or was there, say, excessive test coverage?

Then on a schedule, another agent looks for low-scoring conversations and suggests skill improvements based on them. We've merged quite a few skill changes so far, and the scores help us gauge whether those skill changes are helping.

I'm interested to try another "outer loop" agents focused on code architecture / abstraction and adding static analysis checks. These are even better guardrails for agents than skills to prevent bad code viruses from spreading.

Curious if other people are at setting up supervisor / gardener agents like this.

[Thariq @trq212]

working at a higher level of abstraction has always required understanding the lower level ones, I don't think coding agents changes this

[Matt Roy @themattroy]

@bcherny This is spot on. This is pretty much exactly how I prompt too. That’s why I’m still confused by the huge popularity of some skills repos. Now, you can basically tell the model exactly what’s in your head and get good results.

[Boris Cherny @bcherny]

I am surprised that people are surprised this is how I prompt Claude.

Talk to Claude the way you would a coworker. There's no secret to prompting. There's no need to be overly scaffolded or prescriptive for most tasks -- give Claude a goal, and it will figure it out.

Back in the Sonnet 3.5 days, your prompt mattered a lot. Nowadays, it's much more important to communicate to the model:

1. What you want it to do
2. How much effort you want it to spend
3. How it should verify that it did the right thing

[Garry Tan @garrytan]

I admit my setup for working on gbrain is a bit chaotic. As bugs or issues come in, I spin up a new thread to address it. But Capy's cross-thread coordination and 'virtual standup' conflict avoidance on parallel threads is SOTA

GStack autoplan and eli5 really let me fly

[Matt Roy @themattroy]

Welcome to the moddable era. Hope more apps go this way. Having a blast with it.

[ClaudeDevs @ClaudeDevs]

You can now mod Claude Code:

- Change how it behaves
- Customize the UI
- Swap in your own features

Write one with a few lines of TypeScript, or have Claude build it for you. Mods ship inside plugins, so you install them with /plugin in the CLI or desktop app.

A few examples: