Arize AI

The AI engineering platform for building, evaluating, and improving AI agents & LLM applications. Home of the open source @ArizePhoenix project and Arize AX.

[Arize AI @arizeai]

Jev-as-a-Judge is live in Arize AX.

Ask most LLM judges a yes/no question and you get a paragraph back. Then you parse it to figure out the answer.

Jev returns typed outputs instead: booleans, choices, or rubric scores, with confidence on classifications. You can also evaluate multiple questions in a single call.

Docs --> https://t.co/TaDKpGTx2O

[Arize AI @arizeai]

Who’s checking your evals?

Eval Health is our new managed agent in Arize AX. It reviews your project’s evals and proposes changes to fill coverage gaps, fix broken checks, or remove ones that tell you nothing useful.

It also spots LLM judges you could replace with cheaper code, because sometimes an if statement really is enough.

Get started: https://t.co/gDmKvTs3Wv

[Arize AI @arizeai]

Better visibility, more informed engineering decisions.

[Arize AI @arizeai]

AI cost optimization starts with knowing where costs are coming from.

At TheFork, Arize AX tracing is helping the team get clearer cost signals and identify opportunities to improve efficiency.

Learn how to reduce LLM costs without sacrificing quality: https://t.co/q7PhOM8kjG

[Arize AI @arizeai]

We limited the tools to the logged-in customer's data. The agent requested other customers' data in 20 of 100 runs, but every attempt was blocked. In Arize AX, we set up an online evaluation to flag injection attempts and a monitor to send alerts.

https://t.co/yhfpx975kR

[Arize AI @arizeai]

We built an indirect prompt injection lab to test whether pages could trick an agent into accessing another customer's data. We used fictional customer records. The agent retrieved another customer's data in 28 of 100 runs. We traced and evaluated tool calls in @ArizePhoenix

[Arize AI @arizeai]

Why are we debugging agent runs like diff was never invented?

Trace Comparison in Arize AX puts two traces side by side, with Diff Mode highlighting changed values and added or removed spans.

Try it in Arize AX: https://t.co/EdTMx7a3Dk

[Arize AI @arizeai]

Want to bring these practices into your own AI systems?

Our guide covers a practical framework for building more reliable agents in production.

Read it here: https://t.co/VNS90URTc6

[Arize AI @arizeai]

For @siemenssoftware, trust in GenAI starts with visibility into system performance.

With Arize AX evals and monitoring, the team can spot unexpected agent behavior and investigate issues as they scale.

[Arize AI @arizeai]

Arize AX is now a Claude connector, so “check Arize for my latest traces” actually works.

Connect with OAuth, then query your traces without installing a CLI.

Get started: https://t.co/sTnPK0XHh3

[Arize AI @arizeai]

Not every agent failure shows up in standard logs.

Wurldcoco uses Arize Signal to catch rare edge cases and unexpected tool-calling loops.

See how Signal works: https://t.co/nqFHTp9gTA

[Arize AI @arizeai]

We tested a fallback that lets the chat model choose whenever our tool router's confidence falls below 0.7.

Unfinished replies dropped from 21/300 to 3/300 versus always enforcing the router, although the original agent had zero and the fallback added latency.

Here's how we found the threshold: https://t.co/QJcn5JuGBM

[Arize AI @arizeai]

We improved a shopping agent's first-tool match rate from 78% to 95% by putting a decision model in charge.

Unfortunately, it also produced 21 empty or unfinished replies in 300 turns, compared with zero for the original agent.

Apparently picking the right tool and finishing the job are different skills.

[Arize AI @arizeai]

In our test, a shopper asked an agent to cancel their most recent order.

The tool router leaned toward calling no tool at all, so the agent replied "Let me look up your orders first" and then stopped.

Phoenix's decision spans showed us the alternatives it considered and why it made that choice.

[Arize AI @arizeai]

The engineering behind Alyx's Gemini Enterprise integration is refreshingly straightforward.

A2A handles agent communication, A2UI renders investigation results, and OAuth enforces existing Arize permissions.

Which is good, because nobody needs an AI agent inventing its own access controls.

[Arize AI @arizeai]

Most people who need to know why an AI agent is failing aren't the engineers who built it.

Making them ask an engineer to interpret traces is a pretty weird system.

Our AI engineering agent Alyx is now on Google Cloud Marketplace, bringing trace-backed investigations into Gemini Enterprise.

Learn more: https://t.co/FWfL2T5FMQ

[Arize AI @arizeai]

Giving more people visibility into agent failures is useful. Giving everyone permission to change production agents is a much worse idea.

Alyx in Gemini Enterprise can investigate failures, but prompt edits, experiments, and evaluator changes stay in Arize AX. It's like Cursor, but for your agents.

[Arize AI @arizeai]

But Anthropic's move away from Git gets more interesting when it comes to code review.

Anthropic built Antique to automate much of the review process, so engineers can focus on the intent of changes.

For most teams adopting coding agents, that's almost certainly the bottleneck worth fixing first.

Read more: https://t.co/HGoYr0gT7o

[Arize AI @arizeai]

Anthropic replaced Git for its enormous monorepo after it reached 30,000+ merges and roughly 25 million CI checkouts a day.

That's a very specific scaling problem, and probably not one your team has.

@seldo explains why most developers won't need Scion: https://t.co/xLCjcc6dnj

[Arize AI @arizeai]

Anthropic reports that its Scion setup uses 200x less CPU and memory than the Git mirrors it replaced.

It achieves this by moving commits and history queries server-side and fetching files on demand.

Impressive at Anthropic's scale, but Scion isn't a drop-in Git replacement.

[Arize AI @arizeai]

Why make an LLM read thousands of spans to count errors?

In our benchmark, querying traces with SQL + code mode cost ~17x less than fetching spans page by page.

See how @ArizePhoenix MCP does it, live in Claude Code and Cursor.

Register here:
https://t.co/WQiJ71sbpj