Yabei Z

10,000 hours building with agents. Now working out how to teach my two daughters to command one, and how to tell when they’ve learned it.

[Yabei Z @yabei_8]

@gentschev Same pattern in my logs. On one delegated feature I sampled 20 decisions the agent made: 19 I hadn't known about, 7 weren't what I wanted, none were bugs. Steps vs path reads like one of those. Did your daughter spot it before you did?

[Yabei Z @yabei_8]

@db_bezz Love this. I teach my two daughters to work with AI, and the part I keep watching is whether she can explain it back after the chat closes. A great conversation can feel like learning while the understanding stays in the AI. Did she try one solo after?

[Yabei Z @yabei_8]

Next: I asked r/ClaudeCode how you audit code you can't read. 175k views, 248 comments, and nobody had a way to check what the agent decided.
https://t.co/owJucenngU

[Yabei Z @yabei_8]

Grading my own sessions showed me decisions my agent made that I never did. I figured others might need that too. Before building anything, I asked r/ClaudeCode how you audit code you can't read. 175k views, 248 comments, a split room. A developer said he can't read it either.

[Yabei Z @yabei_8]

I grade my AI coding sessions by how much of each task I decided myself. One feature I'd handed the agent whole: of 20 decisions sampled from it, 19 were news to me and 7 weren't what I wanted. Where I'd revised the design, 10 of 20 were news and 2 I didn't want. None were bugs.

[Yabei Z @yabei_8]

Grading my own sessions had shown me where to look: where I'd said least, the agent had decided most. You don't need to read code to use that. You only need to know where you never said what you wanted. So I thought it might help more than me: anyone building with agents who can't read what gets built.

Before building anything for them, I wanted to know they were out there. So I asked on r/ClaudeCode: I'm not a developer, agents have written about 1.8 million lines for me, how do you audit code you can't read?

It split the room. 175,000 views, 248 comments, and only 62% of votes up. One side said 1.8 million lines can only be patches on patches: have one session write a spec and another rebuild from scratch, or accept that you can't audit it and trust your tools. The other side surprised me. A developer said he can't read the code Claude writes either, so now everyone asks AI to audit AI.

Neither side had a way to check what the agent decided without you. The tools people named, agents hunting for bugs, a security review, CodeRabbit, all look for code that's broken, not for code that works but isn't what you meant.

So the problem isn't only mine, and nobody in that thread had an answer for it. I promised them I'd run what they suggested on my own projects and post what each one found. That's next.

[Yabei Z @yabei_8]

Grading my own sessions showed me decisions my agent made that I never did. I figured others might need that too. Before building anything, I asked r/ClaudeCode how you audit code you can't read. 175k views, 248 comments, a split room. A developer said he can't read it either.

[Yabei Z @yabei_8]

@DemoGap Never coded either. What cost me most wasn't the breakage, it was the quiet choices. On one feature I sampled 20 decisions the agent made: 19 I hadn't known about, 7 weren't what I wanted, none were bugs. Which ate more of your days, the lies or the breaks?

[Yabei Z @yabei_8]

@kaizenbuild Tool. I can't code either. When I counted my messages to coding agents since July, about half were questions: what does this mean, why does it work, are you sure? That's what makes my "no, not like that" land. Without asking, I often didn't know what it had actually done.

[Yabei Z @yabei_8]

@gbrlbldssn Love that it grows one character at a time. One of my daughters plays each new version right away and comes back with very specific asks, like bigger rewards or more food. Who decides what Mercury can do, your kids or the agent?

[Yabei Z @yabei_8]

Next: I checked 20 decisions the agent made on one feature. I hadn't known about 19, and 7 weren't what I wanted. None were bugs.
https://t.co/SDcJzf8qQF

[Yabei Z @yabei_8]

I grade my AI coding sessions by how much of each task I decided myself. One feature I'd handed the agent whole: of 20 decisions sampled from it, 19 were news to me and 7 weren't what I wanted. Where I'd revised the design, 10 of 20 were news and 2 I didn't want. None were bugs.

[Yabei Z @yabei_8]

Being clear with an AI is a skill. Being clear about everything wastes what it knows. A sign-up form it has built a thousand times: give a direction, let it do it its way. Something it has never seen: break your idea into precise pieces. The real skill is knowing which is which.

[Yabei Z @yabei_8]

I'd built a scale that grades each task by how much the person decided rather than left to the AI. The first test was obvious: grade my own sessions, and see whether the grades match anything real in the project.

I read the low tasks closely. They were the ones where I'd said least: a request, and little else. And in the code behind them were rules I had never set. A ranking that ignored anyone not signed in. A follow button that brought a single notice and nothing else. None of it was a bug. It all worked. It just wasn't what I had in mind, or was something I'd never thought about.

The high tasks were the opposite. I could tell you why each part was the way it was.

I checked more sessions the same way and kept seeing it. Where my words were weakest, I knew least about what got built, and that's where I kept finding things I hadn't meant.

The numbers in the post come from one check. The feature I'd handed over had a brief an AI wrote in another session, and once it was built I didn't look at it for four days. For it, and for a part where I'd revised the design, I took 20 of the agent's decisions and answered two things myself: did I know about it, and was it what I wanted.

It explained something I kept running into: testing my own app and finding features I'd never asked for. Code review wouldn't catch those. It asks whether the code does what it's meant to. Nobody was asking whether it was what I meant.

The most dangerous thing in my project wasn't a bug.

[Yabei Z @yabei_8]

@_ultrashot One of my daughters plays a new version right away, then comes back with very specific asks: bigger rewards, more food. Has yours started asking for changes now that she's played it?

[Yabei Z @yabei_8]

Next: Being clear about everything wastes what AI knows. The real skill is deciding what to spell out and what to leave to it.
https://t.co/7uBNPjm5sx

[Yabei Z @yabei_8]

Being clear with an AI is a skill. Being clear about everything wastes what it knows. A sign-up form it has built a thousand times: give a direction, let it do it its way. Something it has never seen: break your idea into precise pieces. The real skill is knowing which is which.

[Yabei Z @yabei_8]

To teach my daughters, I'm building a way to grade how well a request directs an AI. I tested it blind on my sessions, theirs and two adults', with permission. By how often a request got what it asked for, it ranked us as I would, knowing us: me, the two adults, then my kids.

[Yabei Z @yabei_8]

On the scale I'm building to teach my daughters, the first five levels all reward the same thing: deciding more of what you want. Being able to say it clearly is a real skill. But someone who spells out everything, every time, has designed the whole thing themselves, and isn't using what the AI was trained on.

The AI has seen a thousand sign-up forms and a thousand snake games. On those, the best request can be one line: here's the direction, do it the way you know best. On something it has never seen, the opposite holds. You have to take your idea apart and say each piece precisely enough that it can't fill the gaps with whatever's usual.

So the top level, level six, isn't writing every request at level five. It's knowing which kind of thing you're asking for: lean on what the AI knows, and decide for yourself where it doesn't.

That makes level six different from the other five. Most of the time you can't see it in one request: a one-line request might be level one, or level six from someone who knew exactly why one line was enough. It shows up across many tasks, as a habit, and only sometimes inside a single request, like the one in the picture.

[Yabei Z @yabei_8]

Next: I tested the five levels blind on sessions from five people, including my daughters. Here's what matched my expectations, and what didn't.
https://t.co/Gpx0MVrW5s

[Yabei Z @yabei_8]

To teach my daughters, I'm building a way to grade how well a request directs an AI. I tested it blind on my sessions, theirs and two adults', with permission. By how often a request got what it asked for, it ranked us as I would, knowing us: me, the two adults, then my kids.

[Yabei Z @yabei_8]

Why do some AI requests get what you meant? Over two years of coding, time didn't make me better. Deciding more myself did. "Make me a snake game" gets a generic one. Spell out the look, the rules, what it saves: you get yours. I split that gap into five levels to teach my kids.

[Yabei Z @yabei_8]

Before any formal test, I ran the levels over sessions I already had: two of mine, my daughters' work on the cloud setup, and two adults' with 50+ messages there. The model saw only what each person typed and what came back. No names, no ages, nothing about who they were.

It reads two things. How much of each request the person decided, which is the level. And whether the request landed: whether it got them what they asked for, or they had to be asked what they meant or say it again.

On levels, the four of us who aren't me were most often at level 2, with 16% to 31% of requests at 3 or above. I was at 47%. On landing, the order opened up: my daughters 68% and 79%, the two adults 86% and 93%, me 98%. That's the order I'd have given if you'd asked me, and nobody told the model who was who.

It didn't get everything I expected. I'd have bet my older daughter was ahead of her sister. On both measures, she wasn't.

The limits, before anyone asks. Five people. My older daughter has only 44 readings, and my own part is two sessions I picked. A model made every reading, and nobody checked them by hand. Text pasted in from another AI was left out.

What it can't tell me is whether the levels mean anything for results. Two things need testing: whether the same request, written higher, gets people what they wanted more often, and whether people who write higher do better on work they've never done.

[Yabei Z @yabei_8]

@Acci_teacher With my daughters, remaking a game they know has helped. They can compare what the agent built with the original and explain what needs to change. Just getting a working game wasn't enough to tell me they'd learned anything.

[Yabei Z @yabei_8]

@Daniyal77982632 I haven't deliberately changed the habit. I find learning as I build more efficient than understanding everything before I start. Those questions are part of working on things I don't fully understand yet, not just checking the code.

[Yabei Z @yabei_8]

I thought being good with AI meant three things: asking it to build, checking what it built, getting it unstuck. Then I checked my 5,734 messages to coding agents since July. About 52% were questions: what does this mean, why does it work, are you sure? Most advice skips asking.