Hande

@langfuse (part of @ClickHouseDB)
🎓 @TU_Muenchen 🏡 @UniBogazici

[Hassieb Pakzad @hassiebpakzad]

You can also access jev via @OpenRouter now on @langfuse to power your evals

[Hande @handexxk]

There are so many useful signals to collect from your traces in @langfuse like topic, frustration, signs of success...

love @typesafeai making those checks cheaper to run together at scale with Jev!!

[langfuse.com @langfuse]

Jev-powered evals are now available in Langfuse

👉️ Pay 40-400x less than with frontier models
👉️ Run Jev (by @typesafeai) on all production traces without sampling
👉️ Check multiple criteria at minimal extra cost

https://t.co/1RKrhsfYnW

[langfuse.com @langfuse]

Jev-powered evals are now available in Langfuse

👉️ Pay 40-400x less than with frontier models
👉️ Run Jev (by @typesafeai) on all production traces without sampling
👉️ Check multiple criteria at minimal extra cost

https://t.co/1RKrhsfYnW

[Hande @handexxk]

@Sentdex How does your test bench look like? Super excited for the use cases we will all come up with

[Hande @handexxk]

My favorite classifier after BERT

[Annabell @annabellschfr]

https://t.co/ztlSbOBrN6

[Hande @handexxk]

Meet me at the @aiDotEngineer 👀

[Hande @handexxk]

RT @langfuse: @marcklingen sat down to demo all langfuse feature in a single video.

Agent observability, monitoring, evals, prompt managem…

[Annabell @annabellschfr]

https://t.co/ztlSbOBrN6

[Hassieb Pakzad @hassiebpakzad]

Added cost waterfall to @langfuse cost tracking breakdowns. You can now immediately spot what drives your generation cost (and whether caching kicked in or not)

[Hande @handexxk]

@fatih So cool! Can’t wait for your write up and trying Projects

[langfuse.com @langfuse]

the trace timeline now fits any trace on one screen. twelve spans or twelve hundred.

zoom like a map: scroll to pan, pinch or cmd-scroll to zoom both axes, double-click an observation to fly to it. zoomed out, color carries the observation type.

https://t.co/Y0DMvCgKsj

[Hande @handexxk]

@ulusalomer O sırada cursor

[Hande @handexxk]

Grab a template, run it on your traces!

That's really all you need to get going with evals 🚀

Great work by the team on this 👏👏

[langfuse.com @langfuse]

Introducing out new evaluator template gallery!

The right evaluator setup depends on the specific application context and the failure modes observed. But some general evaluation and classification approaches provide a great starting point early on.

Use the templates to:
✔️ Classify common topics of user requests
✔️ Check if retrieved context was accurately used
✔️ Catch user interactions where users disagree

[Hande @handexxk]

@mervenoyann @nvidia Congrats @mervenoyann 🥳🦆

[langfuse.com @langfuse]

A good eval set ≠ a large eval set. Read @lotte_verheyden's latest article on picking the right metrics for your AI application. Now also available on https://t.co/4bH9oOYfwc

[Lotte @lotte_verheyden]

Before you write good evals, you have to pick the right things to evaluate. Main take-aways:
- source metrics from failures you see in traces
- keep the set small and maintainable
- only keep a metric if you'd take action when it moves https://t.co/G2FZjfidrb

[Hande @handexxk]

@apatheticmagpie To many more 🥂

[cognee @cognee_]

How can you do memory integration into any custom agentic workflow?

[cognee @cognee_]

Every debate about MCP versus CLI eventually lands on the same line: "MCP has real auth built in but CLI doesn't" But is that claim really true?

[Vasilije @tricalt]

https://t.co/BFFAtQqewV