Rhesis AI

Open-source platform & SDK for testing LLM and agentic apps. Define expected behavior, generate and run test scenarios, and review failures collaboratively.

[Rhesis AI @rhesisai]

Read the full breakdown: https://t.co/QqvVz5g3sC

[Rhesis AI @rhesisai]

Most conversational AI testing checks if the answer sounds right.

Healthcare needs something stricter: does the answer meet a bar that was defined before the conversation even started.

That's the gap our latest post digs into
#AI #Healthcare

[Rhesis AI @rhesisai]

This closes out our 4-week blog relay: infra → integration → a real-world agent → this one, the highest-stakes case.

If a testing approach holds up for medical advice, it'll hold up for whatever you're building.

[Rhesis AI @rhesisai]

What agent development looks like for most teams right now: 6 roles, 6 tools, feedback scattered across Slack, email, sheets, tickets, meetings, and docs.

No wonder nobody can say if it's actually improving.

🔗 https://t.co/fO7OiiuZOf

#AIagents #EvalOps

[Rhesis AI @rhesisai]

Observability tells you what happened.
Testing tells you what will happen next time.
We just shipped two-way connections: Rhesis can now call your agent directly and run tests against it, not just watch logs after the fact.

https://t.co/Zwli8xkYdt

#observability #agent #AI

[Rhesis AI @rhesisai]

Cloud Run → private Kubernetes clusters, no public API server and access only via WireGuard.

4 months — and the best part isn't the migration itself, it's that secrets never touch values.yaml or Git again (ExternalSecrets + Argo CD GitOps instead). https://t.co/evC829mjO9

[Rhesis AI @rhesisai]

So we're calling the missing discipline EvalOps. Full breakdown + the 4 incidents: https://t.co/WVViPtzRqx

[Rhesis AI @rhesisai]

Everyone says their AI is "production ready."

1,300+ companies running AI in production found out the hard way what that word doesn't cover.

[Rhesis AI @rhesisai]

We pulled together 4 real incidents where that gap was the actual root cause, not a code bug, a place where nobody owned checking the model after launch.

[Rhesis AI @rhesisai]

We tried 4 architectures to deploy an 8B model in production. Only one survived contact with real traffic.

https://t.co/eC1EYuGGOQ

[Rhesis AI @rhesisai]

What does your LLM testing actually look like right now?
A: Manual: someone runs it before big releases
B: Partially automated: some CI, still a lot of manual
C: Pretty solid: tests on most PRs, team uses it regularly D: Full EvalOps: everyone's in the loop,automated,ready

[Rhesis AI @rhesisai]

The teams building this now will be the reference cases in 12 months. Braintrust, LangSmith, PromptFoo… good tools. None of them make EvalOps accessible to the whole team.

That's the gap Rhesis fills. Where is your team on this?

[Rhesis AI @rhesisai]

MLOps gave us the discipline to build models reliably. CI/CD for training, versioning and deployment automation.
EvalOps is the same discipline but for knowing whether your model works, continuous and automated. For every change.

[Rhesis AI @rhesisai]

What EvalOps looks like:
-Tests on every PR
-Experiment diffs before every prompt/model change
-Domain experts in the loop
-OWASP coverage before every security review

[Rhesis AI @rhesisai]

Rhesis 0.9 is built for that transition:
→ Project isolation (hard walls between teams/clients) → OIDC SSO with Keycloak
→ Self-hosted Enterprise Edition

You don't rebuild the setup when you scale. What was the last tool you saw go through this transition?

[Rhesis AI @rhesisai]

There's a specific moment every useful AI tool goes through. 1 engineer sets it up, 5 people use it, IT asks about SSO. A client wants their data isolated.