Built for long-horizon agents.
Monitor agents against your own criteria.
Search and investigate patterns at scale.
Benchmark improvements and regressions before rollout.
https://t.co/EIFfrfR72q
Today, @JudgmentLabs opens for general access.
We believe long running agent systems will generate more exhaust data than the world can currently process.
Hidden within agent traces are rich records of how to make these systems better: what users ask for, when agents stumble, and how agents interact with the external world.
Judgment is the most performant monitoring and improvement layer for long-horizon agents.
We’re excited to see what you build!
Most OTel pipelines, whether on ClickHouse or another backend, store every LLM call’s full context window as-is. TrajectoryStore takes your agent’s LLM calls and stores each message once.
We’re publishing a technical deep dive next week.
New dev-tier orgs get 1,000 free Judgment Units (JUs) in their first month. Sign up today: https://t.co/lXxskIv0uz
Or talk to an engineer to see if it fits your stack: https://t.co/iRfnfUmqp2
Introducing TrajectoryStore.
OTel was designed around spans that each record one small operation, like a single HTTP request or database query. Each span holds different data, so storing every span in full works fine.
Long-horizon agents break that assumption. Every LLM call resends the full conversation so far, and OTel records each call as a separate span with that full input attached. Nothing links one span to the next, so the same context gets stored again on every call. A 200-turn run holds 200 overlapping copies of itself.
TrajectoryStore is a new spec for storing agent traces. Each message, tool result, and model response is stored once, and every LLM call links to what it was sent, so you can still rebuild what the model saw at any call.
What that means for your team:
→ Faster search: keyword search on trajectories scans ~50x less data and returns ~25–40x faster
→ Lower storage costs: 70x fewer bytes stored on our internal coding agent (200+ turns per run), translating to around 10x cost savings for many of our customers
→ One clean trajectory for evals, analysis, and dataset curation
It's the first step in new data infrastructure for storing and searching agent trajectories.
Introducing the Eval Optimizer.
Not every eval needs a frontier model. Judgment now learns your agent trajectories and picks the right compute per judge: code, a decision model, an LLM or an agent.
For one customer, it cut the cost per eval by up to 50×. https://t.co/aUTT97Jtu5
Introducing MCP Observability
When you build an MCP, you write tool descriptions and error messages and hope agents use them well. When they don't, your logs show the error, but not what the agent was trying to do or what it did next.
Now you can follow every agent session on your MCP: what it was trying to do, which tools it called, and what it did when one failed.
Introducing Agent-as-a-Judge: evaluating long-horizon agents beyond 1M tokens.
20%+ of our trajectories exceed 1M tokens, the largest 1B+; too long for an LLM judge. Past 1M, agent judge scored 0.21 higher F1 at 62% lower cost than chunked LLM judges. https://t.co/QoC0Iz1GR2
Judgment Labs -> Judgment Slabs
Kicked off @aiDotEngineer Code Summit on the wall 🧗
Thank you to everyone who came out, and to @cognition and @GoogleDeepMind for climbing with us.
Should we run this back 👀 👀 👀
New dev-tier orgs get 1,000 free JUs in the first month!
Sign up today at https://t.co/oDMntghUxf
Get started with the Slack Agent here: https://t.co/hvZ8TTD0nX
Introducing the Judgment Slack Agent
You can now ask Judgment about ANY part of your JL workspace (data analysis, monitoring, issues, offline tests) without leaving Slack.
Just tag @Judgment with a request like "send me an issue report on our support agent every morning at 9am" and JL will post new issues to your channel, ranked by severity. You can also use it to update your judges or set up new tests, right from the thread.
Interacting with JL and catching silent agent failures has never been easier. Start for free today!
New dev-tier orgs get 1,000 free JUs in the first month!
Sign up today at https://t.co/oDMntghUxf
Get started with actions on here: https://t.co/DvhxiIhwiy
Introducing Judgment Actions
Your judges can now take action automatically when it flags a behavior.
For example, if you’re monitoring a customer support agent for signs of user frustration, your trace judge can alert your team in Slack or by email and save the flagged trace to a dataset for later review. Here's what that alert on a customer support agent looks like.
Is your Jev-as-a-judge, LLM-as-a-judge, or agent-as-a-judge checking for what you care about? Is it making the evaluation decisions you would?
You can now create and align judges with the Judgment Agent. Start by describing what you want to catch, or bring your own rubric.
The hardest part of judge alignment is finding examples that expose where your rubric falls short. The Judgment Agent searches your real traces for challenging cases and tests the rubric against them, surfacing ambiguities for you to review.
Agree or disagree with the verdicts, and explain what it missed. The agent uses your feedback to refine the rubric and align the judge’s decisions with your own before creating it.