We had Claude Code build a tool that prices its own session logs. 2 prompts, 126 seconds, $0.30. The tests passed, but real logs still caught it undercounting. https://t.co/9l41Um8NCo
Starting in plan mode cost 73% more per run ($0.16 to $0.28) in our test, and all 6 runs passed either way. Do you plan first in Claude Code, or just start?
@DesertCybeast@wholemars That’s sick, I have a friend who doesn’t even pay for fsd for most of the year and always gets it before me.
I feel they should do something based on the people who actually usd fsd as a primary
@PawelHuryn How many of the reviewer's findings survive Claude Code's check against the code, roughly? Curious whether the INTRODUCED label cuts the false alarms.
@shmidtqq /clear isn't free on related tasks. On three small tasks in the same files, one continued session with a 9-line CLAUDE.md cost $0.082 vs $0.148 for fresh sessions with a 1,501-word one, because each fresh session re-caches CLAUDE.md.
One long Claude Code chat vs a fresh session per feature, same five features, 3 runs each: both passed all 11 checks in 2 of 3 runs. The long chat averaged $0.204 per build vs $0.326 and finished in 76.3s vs 106.5s, because each fresh session re-read the repo.
@ReignSwords Starting fresh has a real cost too. Building five features in one repo, a fresh session per feature averaged $0.326 a build vs $0.204 in one continued session, with the same pass rate, mostly from re-reading the code each time.
@tastybits And run each setup more than once. On our own task, Sonnet 5.5 at high effort passed 5 of 5 for $0.386 a run, while Opus 5.5 at low effort passed 4 of 5 for $0.461. One run of each could easily have shown a tie.
@theo Have you run it inside a coding agent yet? Benchmarks and agent work split a lot for us: Kimi K3.1 Pro tied Sonnet 5.5 at 5 of 5 on a tricky billing-date task, but averaged $0.127 and 148s a run vs $0.054 and 18s.
@mervenoyann Worth checking edit format before trusting it in an agent loop. When we ran Qwen3.8-27B locally on coding tasks, it got the logic right in all 6 diff runs, but only 1 of its patches actually applied. Did Inkling 2 Mini's edits apply cleanly?
We said a fresh session per feature beats one giant chat. So we tested it: 5 features, 3 runs each. Same pass rate (2 of 3 both ways), but one long chat cost $0.204 a build vs $0.326 and finished in 76.3s vs 106.5s.
https://t.co/gjfZ9xw7Bn
@shirahareads One better: run Sonnet 5.5 alone first and compare. On our 3-step feature, Opus planning + Sonnet building + Haiku searching cost $0.649 a run and passed 2 of 3; Sonnet alone cost $0.283 and passed 3 of 3.
@Sownjuinho Start the loop with a check the agent can't edit. When Opus wrote locked tests and Sonnet built against them, our hidden edge cases passed 5 of 5 vs 4 of 5, but it failed the repo's lint every time, so lock that check in too.
"Max 50" is ambiguous to Claude. In our test, 1 of 3 runs rejected a pageSize over 50 with an error when our spec wanted it capped. Write "cap at 50" or "reject above 50" and say which.
@jdnoc Do you check how much it actually delegates? On our 902-file bug-fix test, Fable 5.2 never started a subagent on its own in 3 runs, so all the work stayed on Fable.
@0xShoopy Measure it before you keep it. When we ran Opus planning, Sonnet building and Haiku searching on a 3-step feature, it cost $0.649 a run vs $0.283 for Sonnet alone, and passed 2 of 3 vs 3 of 3.