Hou C.

Engineering AI features you'd be on call for.

Views are my own, not those of my employer or clients.

[Hou C. @promptdeploy]

@rektcapital flip and flop. flip and flop.

[Hou C. @promptdeploy]

The question isn't "can I ship without all this?"

It's "what failure modes am I comfortable with?"

Because systems can be "up" while users have a terrible time.

What layer makes you most cautious in production AI?

[Hou C. @promptdeploy]

The model is not the system.

When I'm evaluating AI features for production, the model isn't my first focus.

I'm thinking about the 7 layers where things can break:

1. Input validation
2. Context assembly
3. Caching
4. Model call
5. Output validation
6. Fallback paths
7. Observability

[Hou C. @promptdeploy]

For a 10k-request-daily chatbot, I'm asking:

- What inputs am I willing to accept?
- How do I handle partial context assembly?
- What happens when caching serves wrong answers?
- Do my fallbacks actually work under pressure?

[Hou C. @promptdeploy]

Agentic coding makes one thing clear — implementation speed was never the constraint. I can run three workstreams now. The limit is deciding what to build and catching when it's wrong. Faster tools, same judgment bottleneck.

[Hou C. @promptdeploy]

The way I think about it: AI features can fail in ways that seem plausible. The output might sound reasonable, and nothing crashes or lights up red, yet damage can spread silently. By the time I notice, it's not just about debugging—it's about doing cleanup. This is why I'd prioritize understanding the blast radius over just focusing on accuracy scores.

[Hou C. @promptdeploy]

Timely detection and containment are crucial to prevent irreversible failures in AI systems.

"We'll keep an eye on it" makes me uneasy.

What I want to know: How long can this be wrong before I can stop it?

Detection only matters because it enables reversibility.

If detection lags, reversibility collapses.
If stopping takes manual effort, reversibility collapses.

Time to containment > accuracy scores.

[Hou C. @promptdeploy]

They're about containment.

I see hard caps on AI costs not just as budget controls, but as a way to make stopping the system both cheap and emotionally easy.

What makes me cautious is if a system can rack up spend faster than I'd shut it off—that feels like irreversibility.

I'd default to boring kill switches over sophisticated monitoring.

[Hou C. @promptdeploy]

Circuit breakers aren't optimizations.

[Hou C. @promptdeploy]

Running multiple coding agents in parallel is easy. Context-switching between them without losing the thread is the hard part. What workflows are people building here — handoff notes, focus blocks per agent, something else?

[Hou C. @promptdeploy]

Agentic coding lets me parallelize everything in my workflow. Except me. Three agents running concurrently, one human context-switching without dropping state. I'm the single-threaded process in my own concurrent system.

[Hou C. @promptdeploy]

Running three coding agents in parallel sounds like 3x throughput. Then you're context-switching between them every few minutes, rebuilding state each time. Working memory turns out to be the ceiling.

[Hou C. @promptdeploy]

If the abstention rate is basically zero—meaning the system almost never says "I don't know"—that's a risk signal to me.

I'd expect edge cases to cause hesitation. If they don't, it suggests the system isn't confident and is overcommitting.

[Hou C. @promptdeploy]

I'd only want to put RAG in decision paths if it can reliably detect when it can't support the answer—then switch to a fallback we trust.

The failure mode I'm concerned about is unsupported certainty from incomplete retrieval.

[Hou C. @promptdeploy]

That's why the gate matters to me: "Can I support the decision-critical parts with what I actually retrieved?"

If not—I'd want to stop and fall back. I wouldn't want to bluff.

User-visible uncertainty is fine. Silent wrongness is not.

[Hou C. @promptdeploy]

The way I think about it: choosing ML over a *complex* heuristic depends on the complexity.

If the logic is simple and stable, I'd likely keep it. If it's brittle and convoluted, ML might be easier to maintain.

For me, the trade-off is maintainability vs operational risk.

[Hou C. @promptdeploy]

This can become your safety system, but it's happening too late and too expensively.

The question I'd ask is: what does my "I don't know" gate actually look like?

Before showing users anything, I'd want the system to ask:
"Can I support the decision-critical parts with sources?"

Not just "does this sound right," but can I support it—explicitly.

[Hou C. @promptdeploy]

The way I think about it: if my RAG system's abstention rate is basically zero, that's a risk signal. It's not proof of failure—but definitely a signal, especially if query difficulty varies.

[Hou C. @promptdeploy]

Edge cases should cause hesitation. If they don't, I'd worry that the system isn't "confident" but rather overcommitting.

Another signal I'd watch for is human correction patterns:
- Support agents quietly fixing AI answers
- Internal teams overriding the bot
- Policy/legal stepping in later