Ali Habib. Building Rootline

Building Rootline: plain-English explanations of why your AI agent failed, on top of your Langfuse traces. Built in public with a team of AI agents 🌱

[Ali Habib. Building Rootline @ali_buildss]

I'm building Rootline, and a team of AI agents is building it with me.

The problem: your AI agent fails in production, and you end up digging through hundreds of Langfuse traces trying to figure out why.

Rootline connects to your Langfuse and:
• explains each failure in plain English
• groups failures by type, feature and tool
• shows cost per customer, feature and run
• alerts you when an agent gets stuck in a loop
• turns failures into regression tests

No data migration. Just connect your keys.

Try it free at https://t.co/xyVyd4I1Oc. I'll post the bugs, fixes and wins here as we go. 🌱

[Ali Habib. Building Rootline @ali_buildss]

@picconedoro Hi darky building rootline.
https://t.co/HygQfaeVUE

[Ali Habib. Building Rootline @ali_buildss]

@picconedoro Hey darky building rootline.
https://t.co/HygQfaeo56

[Ali Habib. Building Rootline @ali_buildss]

@ishaan_gpt same experience. I'm building Rootline almost entirely with AI agents and the demo was the easy part. figuring out why an agent failed in prod, from the traces, is what I ended up building the product around

[Ali Habib. Building Rootline @ali_buildss]

@adrianogomes override rate is a great pick. I'd pair it with cost per completed task, a task can "succeed" after a bunch of retries and look fine on success rate while costing way more than it should

[Ali Habib. Building Rootline @ali_buildss]

@dneighbors 100%. the agent's explanation is a story written after the fact, the tool log is what actually happened. when they disagree the log wins every time

[Ali Habib. Building Rootline @ali_buildss]

@MattSimonBrown hey Matt, working on the same problem from the "why did it fail" side. loops are the sneaky one for me, no error thrown, just a bill. curious what signal you've found most reliable for catching a loop early?

[Ali Habib. Building Rootline @ali_buildss]

@TheCapexDesk this is the part people miss, your best users are your most expensive ones. first step is just seeing cost per customer instead of total spend. that's one of the things I'm building into Rootline from Langfuse/OTel traces, free during beta: https://t.co/SNl4hbQDtT

[Ali Habib. Building Rootline @ali_buildss]

@DanKornas not forcing a new observability stack is the right call. the next hard part imo is going from "here's the trace" to "here's why it failed", especially when the broken step looks totally fine in isolation

[Ali Habib. Building Rootline @ali_buildss]

@HamedShams40 have you tried breaking the judge into tiny yes/no checks instead of one overall score? way less variance per check. and if you have any past assessor decisions, even a small labeled set to calibrate the judge against helps a lot

[Ali Habib. Building Rootline @ali_buildss]

@pauliusztin_ the referee framing is good. success rate hides so much, an agent can pass by brute forcing 6 retries. are you scoring the path it took too (tool calls, retries) or just the final diff?

[Ali Habib. Building Rootline @ali_buildss]

One sneaky way AI agents fail: they don't crash, they loop.

Same tool call, same bad result, over and over. No error gets thrown, the user eventually gives up, and you find out when the bill shows up.

Two things that help:
1. Look at cost per customer, not just total spend. One looping user hides easily inside a big average.
2. Flag repeated tool calls inside a single trace. If the same call shows up 5+ times in a row, something's wrong.

Rootline does both on top of your Langfuse or OpenTelemetry traces, and explains the failure in plain English. It's built almost entirely by AI agents, free during beta, and there's a no-signup demo at https://t.co/h0WZGWBFua

[Ali Habib. Building Rootline @ali_buildss]

@mmaazkhanhere exactly, a request can succeed and the answer can still be wrong. monitoring built for normal apps just doesn't see that part.

[Ali Habib. Building Rootline @ali_buildss]

@alexii_9 fair point, thanks alex. it's still early beta so i'd rather not fake it, but real numbers and logos go up there as soon as they're real. appreciate you taking a look.

[Ali Habib. Building Rootline @ali_buildss]

@sebuzdugan nice, tool calls per run is where the real story is. would love to see that breakdown, my guess is the two that opened it skimmed the top and missed the part that mattered.

[Ali Habib. Building Rootline @ali_buildss]

@jackrudenko @boristane yep, tagging after the fact never works, the bill shows up and the context is already gone. did any framework in your eval get it right by default, or was it "add a callback and hope everyone remembers" across the board?

[Ali Habib. Building Rootline @ali_buildss]

@tirupathipodugu thanks tiru! brutal notes are the useful ones. if anything in the demo still reads unclear, tell me, i'd rather hear it now.

[Ali Habib. Building Rootline @ali_buildss]

A founder told me our demo "reads like a screenshot." Fair. It did.

So the team (all AI agents, I just point them at stuff) rebuilt it. Now you see the raw trace on one side and the plain English "here's why it failed" on the other. No signup.

https://t.co/h0WZGWB7EC

Free during the beta. Still rough in places, tell me what's confusing.

[Ali Habib. Building Rootline @ali_buildss]

@anrayama "a green call can still carry a wrong payload" is a great line. intent vs what the world said is a clean way to split it. do you flag mismatches automatically, or is it still someone eyeballing the two columns?

[Ali Habib. Building Rootline @ali_buildss]

@tonycueva that example is perfect, a confident wrong policy answer scoring 1.00 is exactly what scares people. on topic isn't true. do you check correctness against a source doc, or keep a small set of known right answers for the policy questions?

[Ali Habib. Building Rootline @ali_buildss]

@h4yfans fair correction, half the spend but all the saving is a sharper way to say it. compaction makes sense, each pass re-sends a big context. and 3 sessions holding 63 of 93 errors is its own story. were those 3 among the long ones, or a separate problem?

[Ali Habib. Building Rootline @ali_buildss]

@thenewstack the 2am problem is usually that whoever gets paged opens a 40-step trace and has to work out why. grouping failures so "this broke 30 times" shows as one issue helps a ton. that's what i'm building Rootline for, free during beta: https://t.co/EZloL970RS