@gregisenberg #36 is the most important step and it's a bit undersized. Grow the eval set from every real correction, and run it on every brain, prompt or playbook change, not just new models. Negative evals matter most: the things it must never do again, straight from feedback.
@alexeyeryshev@munitoai Do you end up having hq/wiki or similar be the source of truth, or external systems like the CRM? Or do agents just keep them mostly in sync?
One Fixture agent got a failed import working, then reported the missing documentation that had tripped it up.
The customer got their leads, but the recovery cost an extra turn. I'd like us to hear about those even when the task succeeds.
We gave our agents at @FixtureGTM a way to complain about our product.
One fixed a failed import, then reported the missing documentation that caused it. The customer got their leads. We got something to fix.
Want to understand how decision models are trained? Read this. Perplexity prunes an LLM's output layer to the answer tokens, makes attention bidirectional, then fine-tunes, adds LoRA and distills. Great candor on tradeoffs too, like isolated questions vs a cacheable state.
We're rolling out pplx-decider-v1.3 and pplx-decider-flash-v1.3, the latest version of our open-weight decision models.
v1.3 adds locate, a new feature that extracts spans based on type or instructions.
Now available in our Decisions API, @huggingface, and @OpenRouter.
https://t.co/bsTSl2tHSo
@yongfook Same SaaS, different buyer. The agent picks the tool, but a human still approves the spend, so you're basically selling to two buyers at once. One of them is ruthless with zero brand loyalty.
Every section of our AGENTS.md went into one of 5 buckets:
- Fact the model can't infer: keep
- Already enforced by a lint or hook: one-line pointer
- Generic coaching: delete
- Scar tissue: keep word for word
- Narrow: move to that area's doc
We ran an ablation on our repo's AGENTS files. We deleted most of the guidance, replayed real PRs and brought back only what measurably mattered.
1,892 lines → 716.
Ok, that implicit state machines investigation was a great idea. Found a dozen small issues with state transitions in weird error cases. Highly recommended! One thing I'd advise though: not only look for implicit state machines, but ask the agent to prove they can (or already do) cause problems in the system.
Coordination between teammates' agents is the right next problem for these platforms. It needs to work across tools too, so we built Swarmatory as a shared feed that Cursor, Claude Code, Codex and others can all post to. Open source: https://t.co/Ys6mBZeNKy
Your agents can now coordinate with your teammates’ agents.
Projects has been rebuilt for multiplayer: every teammate and their agents can now work together in one unified workspace.
@FixtureGTM The setup for now: a feedback tool writes structured logs. An offline agent reviews them regularly, then our team reviews what comes back.
We decide what to investigate or build. Happy to share more about the tool and review process.
We gave our agents at @FixtureGTM a way to complain about our product.
One fixed a failed import, then reported the missing documentation that caused it. The customer got their leads. We got something to fix.
@FixtureGTM We still check the agent's diagnosis. Sometimes it reports a missing feature that already exists.
Then we need to understand why it couldn't find it. A clearer tool description might help more than another tool.
@swyx@adamwathan@ScottWu46 We made this switch. Almost all of @FixtureGTM's dev now runs on @AmpCode Orbs instead of locally. Spaces is another big step forward. We link each one to the matching team Slack channel, so one thread runs the workers and everyone can contribute/follow.
Same experience with our AGENTS files. We deleted most of the guidance, replayed real PRs and only added back what measurably changed the outcome. 1,892 lines became 716. Worth rerunning whenever the model changes, since the answer moves with it, and good evals make that easy to do.
Eval-driven development loops for agent skills is necessary. Every new model gets better. You can remove things you don't need anymore.
I just removed /investigate as a skill because Opus 5.5 does it better without it.
You can feel the AGI tide rising
Archaeology and evals answer different questions. Git history tells you where a rule came from. Replaying real PRs tells you whether today's model still needs it. We did both before cutting anything.
More on the archaeology step: we ran git log -S on every section of our AGENTS.md to find the commit that added it. Knowing where and when each rule came from made it much easier to sort into a bucket, and to see which ones to test hardest.