MLflow

The open source developer platform to build AI applications and models with confidence.

[MLflow @MLflow]

🔹 Full sessions grouped together, so long conversations stay debuggable
🔹 MLflow 3.12 tracing for Claude Code, Codex, Gemini CLI, OpenCode, Qwen Code, and OpenHands

🎥 Full webinar: https://t.co/JRbV5uQ4X0

#MLflow #CodingAgents

[MLflow @MLflow]

MLflow 3.12 deep dive clip: why coding agents need tracing 👇

Yuki Watanabe walks through what shows up in the trace when you turn it on:
🔹 Every turn, tool call (Read, Bash, Edit), and sub-agent step
🔹 Token usage and latency per span, including cache breakdown

[MLflow @MLflow]

Genie fixes from failed evals 👇
🔹 Traces + space config in
🔹 LLM suggests concrete edits
🔹 Shorten signal-to-patch time

📕 Read the cookbook: https://t.co/ZhYV7LpdFX

#MLflow #GenAI #Genie

[MLflow @MLflow]

MLflow 3.13.0: RBAC + Admin UI for self-hosted servers 👇

🔐 Roles as reusable permission bundles
🖥️ Admin UI (no REST endpoints)
📦 Experiments, models, prompts, scorers, Gateway

Release highlights: https://t.co/JNPgjyWXrU

#MLflow #MLOps

[MLflow @MLflow]

MLflow 3.13.0 is a major update that runs AI observability at scale, focusing on access control, the lifecycle of your trace data, and richer support for agents. 🙌

🔗Check out the highlights of the release: https://t.co/JNPgjyWXrU

#mlflow #opensource #linuxfoundation

[MLflow @MLflow]

LLM judges for Genie traces 👇
🔹 Built-in baseline judges
🔹 Custom SQL/semantics checks
🔹 Start on highest-risk traces

📕 Read the cookbook: https://t.co/DGmpxHU5xe

#MLflow #Genie

[MLflow @MLflow]

Thousands of traces, no systematic way to spot bad agent runs. MLflow Automatic Issue Detection 👉 choose CLEARS categories, run analysis in three clicks, triage issues in the UI.

🔗 Learn more: https://t.co/a8gNos0vs7

#MLflow #LLMOps #GenAI

[MLflow @MLflow]

Trace + eval Genie in MLflow 👇
🔹 Full Genie pipeline
🔹 MLflow traces + judges
🔹 Tighten one pilot space first

📕 Read the cookbook: https://t.co/kd9fQxwSof

#MLflow #Genie

[MLflow @MLflow]

Vibe-checking works until it doesn't. Change one prompt, break three behaviors—and you can't tell if you moved forward or backward.

Eval-driven development in MLflow 👇
1️⃣ Trace — mlflow.openai.autolog() + @mlflow.trace spans (latency, tokens, cost)
2️⃣ Evaluate + prompts — mlflow.genai.evaluate(), make_judge(), Prompt Registry, optimize_prompts (GEPA)
3️⃣ Prod — same judges on live traces; agent dashboards for cost/latency/quality

🔗 Learn more: https://t.co/I4zS7unOrn

#MLflow #LLMOps #GenAI

[MLflow @MLflow]

Right answer, wrong trace? MLflow + TruLens Agent GPA scorers read the full span tree 👇

🔹 10 TruLens scorers: 6 Agent GPA + 4 RAG
🔹 95% agent errors on TRAIL vs 55%
🔹 mlflow.genai.evaluate() w/ RAG + Phoenix

🔗 Read more: https://t.co/7jIlkg3VRO

#MLflow #TruLens #GenAI

[MLflow @MLflow]

Red-team LLM apps in MLflow 👇
🔹 Adversarial eval inputs
🔹 Safety scorers + guidelines
🔹 Rerun after model/prompt changes

📕 Read the cookbook: https://t.co/VffLiylPJZ

#MLflow #GenAI

[MLflow @MLflow]

Claude Code can burn through dozens or hundreds of LLM calls in one session. MLflow 3.12.0+: route it through AI Gateway with two env vars for traces, budget alerts/limits, and guardrails. No SDK changes.

🛣️ Setup: mlflow server → Gateway endpoint → ANTHROPIC_BASE_URL to the claude-code proxy. Run claude as usual.

Learn more 👉 https://t.co/2xSoVXuJZ2

#MLflow #AIGateway #ClaudeCode

[MLflow @MLflow]

RAG eval end-to-end in MLflow 👇
🔹 Trace retrieve + generate
🔹 Built-in retrieval/gen judges
🔹 Localize failure to a stage

📕 Read the cookbook: https://t.co/5JTKfuX3xt

#MLflow #RAG

[MLflow @MLflow]

Catch this session at Data + AI Summit (June 15-18, SF)! 🌟

Agent quality via vibe-checking breaks at scale.
🔁 MLflow self-evolving test harness
🧪 Bad-answer feedback → automated tests
✅ Coding-agent fixes vs. accumulated suite

🎤 Adam Gurary & Yuki Watanabe

Session details: https://t.co/8AhnjLPmEP

#MLflow #DataAISummit

[MLflow @MLflow]

Prompt lifecycle in MLflow 👇
🔹 Registry-backed versions
🔹 Eval-gated promotion
🔹 Rollbacks without guesswork

📕 Read the cookbook: https://t.co/omRMtW9UIt

#MLflow #GenAI

[MLflow @MLflow]

.@OpenHandsDev agents edit files, run commands, and browse the web on their own—but there’s no structured record of what happened or whether the result was good.

MLflow connects via @opentelemetry to trace every step, evaluate runs with built-in judges, and route model traffic through AI Gateway for budget and usage control.

Learn more 👉 https://t.co/YGQlhIB7yK

#MLflow #OpenHands

[MLflow @MLflow]

@OpenHandsDev agents edit files, run commands, and browse the web on their own—but there’s no structured record of what happened or whether the result was good.

MLflow connects via @opentelemetry to trace every step, evaluate runs with built-in judges, and route model traffic through AI Gateway for budget and usage control.

Learn more 👉 https://t.co/YGQlhIAzJc

#MLflow #OpenHands

[MLflow @MLflow]

New on the MLflow channel: evaluate a RAG agent end-to-end with Joana Mesquita, MLflow Ambassador 👇

📌 Prompt Registry + production aliases
🔍 Traces with SME ground truth
⚖️ Ragas, Phoenix + custom LLM judge

Watch now: https://t.co/ryZbnFHONj
Blog: https://t.co/HIMMCatGcv

#MLflow #RAG

[MLflow @MLflow]

New on the MLflow channel: evaluate a RAG agent end-to-end with Joana Mesquita, MLflow Ambassador 👇

📌 Prompt Registry + production aliases
🔍 Traces with SME ground truth
⚖️ Ragas, Phoenix + custom LLM judge

Watch now: https://t.co/ryZbnFHONj
Blog: https://t.co/HIMMCatGcv

#MLflow #RAG