Flying Higher AI

Applied AI engineering & automation. Building, evaluating, and benchmarking RAG systems, workflows, agents, and decision models. Exploring what works.

[Flying Higher AI @flyinghigherai]

I started Stage 3 with 11 fault injections: API timeouts, broken JSON, substring spoofing, HTTP 500s, threshold edges, NaN/null scores, and expired policies. They exposed a JSON-escaping bug and a Switch-boundary overlap. After fixes, all 11 passed with zero false auto-sends.

[Flying Higher AI @flyinghigherai]

Before Stage 3, readers helped expose architectural blind spots in my earlier retrieval work. I turned their critiques into concrete tests. These were the failure modes I wanted the pipeline to survive:

[Flying Higher AI @flyinghigherai]

One interesting note on Google’s Gemini Decision 1 Model is that the input types include text, images, audio, video, and PDFs, whereas JEV only supports text and images.

[SG Reflections @saharg6683]

I created this with Claude Sonnet just for fun. It’a 15 second video to show a RAG pipeline.
Actually I had this idea to write a fantasy story that came to my head while doing experiments in vector databases and RAG mechanisms.
And I made this to go with it. Story: soon.

[Flying Higher AI @flyinghigherai]

@n8n_io - I did this on n8n that I setup in Codespace on Github. You might be interested to see what I did/Full technical write-up in the link.

[Flying Higher AI @flyinghigherai]

Can a RAG pipeline survive a 40.9-point drop in retrieval accuracy? I put VeriRoute AI through a 150-query holdout stress test. Here's what happened.
https://t.co/bWnQ303Xwr

[Flying Higher AI @flyinghigherai]

What happens to your RAG pipeline when an API times out at 1ms, a reranker passes a NaN score, or an LLM returns broken JSON? Does it fail closed and hold the ticket, or does it accidentally auto-send an unverified draft to the customer?

Full technical breakdown coming soon…

[Flying Higher AI @flyinghigherai]

Full technical write-up:
https://t.co/bWnQ304vlZ

[Flying Higher AI @flyinghigherai]

I started Stage 3 with 11 fault injections: API timeouts, broken JSON, substring spoofing, HTTP 500s, threshold edges, NaN/null scores, and expired policies. They exposed a JSON-escaping bug and a Switch-boundary overlap. After fixes, all 11 passed with zero false auto-sends.

[Flying Higher AI @flyinghigherai]

Before Stage 3, readers helped expose architectural blind spots in my earlier retrieval work. I turned their critiques into concrete tests. These were the failure modes I wanted the pipeline to survive:

[Flying Higher AI @flyinghigherai]

Insight 3: Confidence gating contained the damage. Tier 1 precision stayed at 88.9%; auto-sends dropped from 81.6% to 26.7% without code changes. Ambiguous tickets moved to human review, and the Verifier caught an invented 48-hour SLA.

[Flying Higher AI @flyinghigherai]

Operational alignment: Retrieval quality is only part of the problem. Each escalation path also needs a defined queue, SLA, and cost ceiling.

[Flying Higher AI @flyinghigherai]

Before Stage 3, readers helped expose architectural blind spots in my earlier retrieval work. I turned their critiques into concrete tests. These were the failure modes I wanted the pipeline to survive:

[Flying Higher AI @flyinghigherai]

Escalation loops: Bad or corrupt inputs must not bounce endlessly between tiers. The workflow needs a hop cap and a clear fallback when routing fails.