Custom classifiers are available in Tessary Paid, through self-hosting or managed hosting. Message us to try them.
Explore Tessary: https://t.co/bn1VD0y8BS
The classifier monitors every new session, using only the relevant context and a small decision model to keep checks cheap.
When it detects a recurring issue or a change in behaviour, Tessary validates the finding and investigates the root cause with your agent codebase.
It then uses a stronger teacher model to calibrate a smaller, cheaper decision model against your real traces.
You review examples to check that the judge matches your intent, without writing code or manually labelling a whole dataset.
Tessary’s authoring agent helps you turn that intent into a classifier.
It finds relevant examples from your real traces and guides you through labelling a few to clarify exactly what the classifier should catch.
Our built-in classifiers cover common issues like tool errors, frustrated users, and ungrounded answers.
Your agent may also promise refunds it cannot give or respond in a tone that doesn’t fit your brand. These failures depend on what your product is meant to do.
We’ve added custom classifiers to Tessary. You can now describe a failure specific to your agent and have Tessary build a classifier to monitor it in production.
Tessary is now available on Cloud. You can try our open-source agent reliability platform without self-hosting.
Tessary monitors every production trace, detects issues that sampled evals miss, and investigates to find the root cause.
You can self-host Tessary and run the model on your own hardware. Paste this into your coding agent:
“Self-host Tessary for me by following https://t.co/IM1ROlq9q8”
We’ve written about how we trained and evaluated the model, and why Tessary looks for an increase in flagged answers rather than opening a finding for every flag.
Read the blog post: https://t.co/hnpXjx2HMy
When Tessary detects an increase in frustration, it opens a case with the affected conversations and runs root-cause analysis with your agent codebase to identify the cause.
Read more: https://t.co/4H3AzpdD3p
Self-host Tessary: https://t.co/6h8JVdYnxN
We’ve added a frustration classifier to Tessary.
It finds conversations where users are unhappy with your agent, detects when frustration increases, and helps identify what is causing it.
We then tried TypeSafe’s Jev.
At a 6% false-alarm rate, Jev reached 68.4% recall and 85.6% precision. Sonnet 5 posted basically the same numbers while costing ~40x more.
That made it practical to run the classifier across every eligible conversation.