Lee Ryeong

Korean indie hacker 💪
Gym + Code every day
Obsessed with digging deep into AI security
— one deterministic proof at a time.

[Lee Ryeong @OHS1327]

Same code. Different models.
Leak rate: 0% vs 90%.

I threw prompt injections at my self-hosted AI agent…
and the results were shocking.

System prompt + API keys leaked at wildly different rates depending on the backend LLM.

Tested 5 different models with the scanner I’m building in public.

Full write-up + detailed results here 👇

🔗 Indie Hackers: https://t.co/ihMew39OdG

GitHub: https://t.co/kfDGCnxd4Q

#BuildInPublic #LLMSecurity #AIagents #PromptInjection

[Lee Ryeong @OHS1327]

Simpler than it probably looks — it's counting, not a formula. I plant a fake "canary" secret in the agent's system prompt, send probes, then scan two surfaces separately: the final answer and the reasoning trace. A deterministic pattern-matcher (no AI judging) counts how many runs the planted secret literally shows up in each. So a result reads like "surfaced in the reasoning X of N runs." The scan code is open if you want to run it yourself — repo in my bio / the post. Happy to point you to the exact file.

[Lee Ryeong @OHS1327]

New: I tested self-hosted AI agents for credential leaks — scanning both the answer AND the hidden "reasoning" trace.

Answer leaked a planted key ~0.5% of the time.
The hidden reasoning leaked it ~26% (one model: 74%).

A final-answer-only detector undercounts credential exposure ~50×.
[https://t.co/kfDGCnxd4Q]
[https://t.co/CvMPYkD5Kf]
[https://t.co/OHlvgZw0iY]

#AIAgent #AISecurity #LLMLeak #CredentialLeak #PromptInjection

[Lee Ryeong @OHS1327]

That makes complete sense.

I’ve been moving in the same direction — preferring a separate validation step rather than relying on command-level guardrails alone. Letting the agent decide its own trustworthiness on security-sensitive changes feels risky. I want the workflow to explicitly fail closed unless a clear, verifiable artifact is present.

Have you found any lightweight patterns or hooks that make implementing this separate validation step practical across different projects?

[Lee Ryeong @OHS1327]

The audit paradox for AI agents: compliance wants a trail, security says don't retain
raw reasoning (it's where secrets surface). Way out: audit the detection, not the
reasoning — and keep a fingerprint (type + scope + salted one-way hash), never the
value. Credential-free record, audit still answerable.

Pattern + honest limits + where I shipped it
🔗 [https://t.co/747QnZIRXx]
Repo (Apache-2.0): 🔗 https://t.co/kfDGCnxd4Q

#AISecurity #AIagents #LLMSecurity #AgentProof #PromptInjection #IndieHacker #BuildInPublic #CyberSecurity

[Lee Ryeong @OHS1327]

Exactly. That's the core risk.

Fable 5 can make a bad change sound completely reasonable in natural language. That's why I force every security-sensitive edit to produce a concrete artifact — preferably a failing test or a clear diff + log. If there's no hard evidence, the change doesn't go through.

The "red test refuses to negotiate" part really resonates. How are you enforcing that boundary in practice?

[Lee Ryeong @OHS1327]

It really is!

Fable 5 has completely changed how I approach building self-hosted AI security agents. The combination of strong reasoning + structured outputs (especially forcing diffs, logs, and tests on security-sensitive paths) has made reflection loops far more reliable and helped surface credential issues much earlier.

Changing lives might be an understatement at this point 😂

[Lee Ryeong @OHS1327]

Agreed — this went somewhere real. The complementarity is the keeper: detective catches the leak that never became an action, preventive catches what never got logged, and each is blind exactly where the other sees. I'm writing up the detective/fingerprint side now — the preventive framing is yours to tell better than I could. Good exchange; thanks for pushing on the precise distinctions.

[Lee Ryeong @OHS1327]

If you audit your AI agent by reading its answers, you're watching the wrong surface.

In 2026 the leak is often in the model's reasoning, not the reply — and hardening
the output doesn't secure the thinking. I measured it deterministically on 4
lightweight models.

The cost isn't abstract: a stolen key burned $82K in 48h for a 3-person startup this
year.

Full writeup — findings + the honest limits of my scanner:
🔗 https://t.co/ZbEIuJxlv1

Repo (Apache-2.0), incl. the defense prompts:
🔗 https://t.co/kfDGCnxd4Q

#AISecurity #AIagents #LLMSecurity #AgentProof #PromptInjection #IndieHacker #BuildInPublic #CyberSecurity

[Lee Ryeong @OHS1327]

This is really inspiring.

I’ve been experimenting with Fable 5 for building structured agent systems, and seeing how quickly people are shipping actual games with it (gameplay-first, not graphics-first) makes me want to try applying similar structured workflows to game dev as well.

Great to see more indie builders succeeding with solid gameplay. Keep going!

[Lee Ryeong @OHS1327]

This is seriously impressive!

Fable 5 keeps showing how powerful it is for rapidly building complex, interactive systems. The ability to go from idea to a walkable 3D London environment with physics in just a day is exactly why it’s so valuable for agentic workflows.

Great demo — bookmarking for inspiration on structured creative building with agents.

[Lee Ryeong @OHS1327]

This is exactly the kind of artifact coding agents need.

For the self-hosted AI agents I’m building, having a clean, consumable SKILL.md (or equivalent) for specific domains has been incredibly effective. It improves consistency, reduces unnecessary exploration, and helps maintain deterministic behavior — especially useful when layering in security checks and reflection loops.

Great addition to the book. Bookmarking the SKILL.md.

[Lee Ryeong @OHS1327]

Excellent comprehensive roadmap!

For the self-hosted AI security agents I’m building, strong SQL fundamentals are surprisingly practical — especially for building audit logs, querying security events, managing agent memory/state, and handling data in reflection loops.

The sections on optimization, transactions, and real-world projects look particularly useful. Bookmarking this for reference.

[Lee Ryeong @OHS1327]

That’s exactly the kind of disciplined pattern that actually works in practice.

I’ve been enforcing something very similar on security-sensitive paths in my agents: narrow scope, mandatory artifact (diff/log/test), and no proceeding without evidence. It’s caught multiple reflection loop and credential-handling issues that softer “just review and advise” approaches would have missed.

Do you apply the hard stop at the command level itself, or through a separate validation step afterward?

[Lee Ryeong @OHS1327]

Exactly. Practicing failures proactively is the only way to build agents that stay reliable under real conditions.

For the AI security auditing agents I build, I regularly inject controlled failures (malicious tool calls, network issues, prompt variations) while enforcing security invariants. It’s caught several potential credential leaks and unauthorized actions that would have been missed otherwise.

Completely agree — failure practice before production is non-negotiable.

[Lee Ryeong @OHS1327]

That's a great starting point — breaking dependencies first really exposes weak points in the system.

For the security auditing agents I build, I often start by mocking or breaking tool call dependencies and external service connections. It helps surface issues around credential handling and unauthorized actions early, before layering in malicious inputs or prompt variations.

Do you focus on specific dependency types (like LLM APIs or tool integrations), or do you go broader?

[Lee Ryeong @OHS1327]

Excellent resource!

For the self-hosted AI security agents I'm building, a well-organized collection of red teaming tools like this is incredibly useful. Quick access to recon, privilege escalation, and testing tools will help a lot when designing agent-driven security workflows and simulations.

Bookmarking this. Thanks for sharing!

[Lee Ryeong @OHS1327]

Fair — that's the cost I'll concede on the detective side. Once you retain a trace to scan it, the log's integrity becomes a second thing to defend: tamper-before-scan is real. You can shrink that surface — scan-then-discard, keep only a credential-free fingerprint — but not erase it; the record still needs integrity if it feeds audit. Preventive sidesteps the whole regress by depending on nothing kept. Genuine structural edge for your side. Mine still catches the leak that never becomes an action — at that retention cost. Asymmetric tradeoffs, not redundant.

[Lee Ryeong @OHS1327]

Agreed — recomputability is the sharp distinction. Yours is preventive and capture-independent: the decision node is always there to re-check. Mine is detective and capture-dependent — I can only scan the reasoning if the trace was retained, which is exactly the operational limit I keep flagging. So each reaches a failure the other can't: a leak upstream that never gets logged (yours), vs a secret surfacing in reasoning that never becomes an action (mine). Complementary, like you said — different failure modes, different points in the pipeline.

[Lee Ryeong @OHS1327]

Great comprehensive list!

For building self-hosted AI security agents, strong DSA fundamentals are surprisingly practical. Topics like Graph algorithms (shortest path, topological sort), Dynamic Programming, efficient data structures (Hash Tables, Tries, Segment Trees), and Bit Manipulation come up often when implementing attack path analysis, optimized memory handling in agents, or efficient reflection loops.

Solid foundation for anyone shipping production-grade agent systems.

[Lee Ryeong @OHS1327]

Super useful resource!

Even from a software/AI agent perspective, understanding the hardware bottlenecks (especially actuators, reducers, and cost structures) is really valuable. When designing agents that might eventually interface with or control physical systems, knowing these real-world constraints helps build more grounded and reliable architectures.

The interactive breakdowns and live spec sheets look perfect for quick, visual learning. Bookmarking this!

[Lee Ryeong @OHS1327]

좋은 정리 감사합니다!

Fable 5로 self-hosted 보안 테스팅 에이전트를 만들 때 특히 유용할 것 같아요.
“충분한 정보가 있으면 바로 실행” + security-sensitive 작업은 반드시 diff/log/failing test로 증거를 남기라는 조합이, reflection loop 안정성과 credential leak 방지에 정말 효과적이었어요.

공식 가이드 핵심만 이렇게 깔끔하게 뽑아주셔서 바로 적용해보기 좋네요. 공유 감사합니다!