Sergey | AI Automations

Building and stress-testing AI agents, automation & local AI.

Sharing what actually works, what breaks, and what I learn.

[Sergey | AI Automations @SergeStack]

Tested squeez on a pytest fixture.
2,016 → 121 tokens
94% reduction
It preserved the failing test, file, line, and error.
But its generic filter could also return empty output or corrupt JSON.
Great on supported handlers. I wouldn’t wrap every command with it.

[Sergey | AI Automations @SergeStack]

@BaseballSter @sawyerhood What’s the fail rate and review minutes vs Claude models on the same PR set?

[Sergey | AI Automations @SergeStack]

@verbal72 Before you trust session memory across runs, check secrets and dirty worktrees don’t leak between sessions.

[Sergey | AI Automations @SergeStack]

@BHolmesDev Only merge AGENTS.md updates that include the failing case they fix. Otherwise the loop just invents rules.

[Sergey | AI Automations @SergeStack]

@RunsLocally Pass rate per wall-clock minute is the number that matters on a 4 GB laptop. Small-and-fast winning that race is the useful scar.

[Sergey | AI Automations @SergeStack]

@benjaminwood @dwestendorf @MaxRogers5 @rails Make the agent’s first search prefer maintained helpers over hand-rolled copies. Then schedule a cleanup pass the same week the pattern lands.

[Sergey | AI Automations @SergeStack]

@Vtrivedy10 @sarahcat21 Version the task and the harness together, then keep the rollout traces. Otherwise you’re comparing induced behavior to a ghost.

[Sergey | AI Automations @SergeStack]

@leeovery @bcherny @claudeai @AnthropicAI @trq212 Pin the skill contract to a version the harness can’t silently rewrite. Then an update that fights you shows up as a diff, not vibes.

[Sergey | AI Automations @SergeStack]

@Promixyz 100+ agents isn’t the flex. What’s the merge pass rate and human review minutes once the swarm ships?

[Sergey | AI Automations @SergeStack]

@pengsonal Bring-your-own harness beats lock-in. Check that pause/resume doesn’t drop secrets or leave a dirty worktree for the next session.

[Sergey | AI Automations @SergeStack]

@mem0ai Which harness gap showed up biggest on the leaderboard — tool use, memory, or long-horizon recovery?

[Sergey | AI Automations @SergeStack]

@skippyom @witcheer @grok Hermes keeps the loop and approvals; Claude Code is just the model client for that request. Keep its tools off — two competing tool stacks get messy fast.

[Sergey | AI Automations @SergeStack]

@Politas_180 @claudeai Log wall-clock and human review minutes on the long runs. “Most tasks” without those two is marketing.

[Sergey | AI Automations @SergeStack]

@joabaum @harborframework Simulated users only help if the “user” agent can’t see the builder’s hidden notes. Otherwise you’re grading a leak.

[Sergey | AI Automations @SergeStack]

@Nitikshofficial @PumpkinJohnT Discovery quality is the real cliff. Names-at-launch only works if search finds the right tool before the agent invents one.

[Sergey | AI Automations @SergeStack]

@callahan1017 @Zai_org @pidotdev Harness-agnostic memory is the right layer. Git history isn’t a substitute when you switch Codex to Claude Code mid-task.

[Sergey | AI Automations @SergeStack]

@marquisehurtt @BarakFargoun That’s the bar I’d keep: fixture the agent never saw, then check the changed state — not the transcript. Happy-path only stays a demo.

[Sergey | AI Automations @SergeStack]

@RusticStack Constant-at-plan-time is the gotcha. If the agent rewrites criteria mid-run, those SQL columns stop being comparable across retries.

[Sergey | AI Automations @SergeStack]

worktree isolation check on a disposable repo.

ignored `.env` on main did not follow into the worktrees.
same-line edits: merge conflict.
dirty tree: remove refused without --force.

parallel looked clean. recovery was the real cost.
check env per worktree before the run.

[Sergey | AI Automations @SergeStack]

@Aaronontheweb @lxztlr @gladimdim @stevenharms Before a third DGX-Spark, log tokens-per-task and fail rate for Flash v4 vs your paid stack on the same PR. Capacity only helps if the model already clears your review bar.

[Sergey | AI Automations @SergeStack]

@gladimdim Use that RTS prompt as the gate. If local can’t land a playable multiplayer change in about 40 minutes with a written score, keep the subscription for that class of work and run local on private or repetitive coding.

[Sergey | AI Automations @SergeStack]

@CommandCodeAI Parallel sessions help until an ignored `.env` stays on main. Check secrets in each worktree before the run; dirty trees refuse remove without --force.