lejars.g

Local AI, measured not claimed. Luna, a French assistant on Qwen3 14B Q4_K_M via llama-server, RX 9070 XT 16GB, ~52 tok/s (ROCm). No cloud AI model.

[lejars.g @g_lejars]

Luna Orbe Orion : une IA 100 % locale, née au pied du volcan 🌋
Chaque réponse doit être prouvée avant d'être affirmée. #LocalLLM #Ollama #AMD #974

[lejars.g @g_lejars]

I asked my local AI assistant to create a file.
It did it. Without asking me first.

The policy said "confirm". The code path skipped it.

Found it by testing live, not by reading the config.
Fixed. It now asks, waits for "yes", and "no" cancels.

Config ≠ behavior. Test it.

#LocalLLM #AI #Qwen #llamacpp #SelfHosted #AIAgents #Linux

[lejars.g @g_lejars]

Update: fixed and tested live on my machine.
✅ Explicit "create a file" requests now ask for confirmation first (yes = writes once, no = cancels)
✅ Removed an open sudo wildcard rule, replaced by a root-owned wrapper that validates package names
Still open: Luna runs under my own user, and her guardrail files aren't read-only for her yet.

#LocalLLM #AIAgent #SelfHosting #Infosec #Qwen3

[lejars.g @g_lejars]

Luna runs locally, but she's not sandboxed. I just audited her.

✅ Web search
✅ RAM/CPU/temp monitoring
✅ Downloads folder scan
✅ Real-time detection
✅ Security updates (automatic)
✅ File edits inside my home folder

🔒 Kill / quarantine / write_file ask me first
🚫 No delete tool

⚠️ Guardrails live in code, not enforced by the OS
⚠️ Runs under my own user

Audit found gaps. Fixing them next.

#LocalLLM #AIAgent #llamacpp #Qwen3 #ROCm #SelfHosting #Infosec

[lejars.g @g_lejars]

What actually runs my local assistant Luna, checked on my own machine:

llama.cpp's llama-server → Qwen3 14B (Q4_K_M) → one RX 9070 XT, 16 GB VRAM.

32K context, flash attention, q8_0 KV cache. No cloud model. 🌙

#LocalLLM #llamacpp #Qwen3 #AMD #SelfHosted #AI

[lejars.g @g_lejars]

Same GPU, two backends, opposite winners. RX 9070 XT, llama.cpp b1177, Qwen3 14B Q4_K_M:
• Generation: Vulkan +9% at 1k ctx, +23% at 30k
• Long prompt reading: ROCm 13–20% faster
Which one do you run on AMD for long context?
#LocalLLM #Vulkan #ROCm #AMD #Qwen3 #llamacpp

[lejars.g @g_lejars]

Same card, same model: Vulkan vs ROCm on my RX 9070 XT. Vulkan generates up to 23% faster at 30k context, ROCm reads long prompts 13-20% faster. Different workload, different winner. Numbers and scripts: https://t.co/nvGstaULHa
#LocalLLM #llamacpp #ROCm #Vulkan #AMD #Qwen3

[lejars.g @g_lejars]

@richgel999 Backend choice matters too. RX 9070 XT, llama.cpp, Qwen3 14B Q4_K_M, ROCm 7.2.4: Vulkan (RADV) generates 9% (1k ctx) to 23% (30k) faster, while ROCm reads long prompts 13–20% faster. #ROCm #Vulkan #llamacpp #LocalLLM https://t.co/54v1sT9rrP

[lejars.g @g_lejars]

Nice project! One thing I measured on an RX 9070 XT (llama.cpp b11177, Qwen3 14B Q4_K_M): the backends win at different stages. Vulkan generates faster (+9% at 1k ctx, +23% at 30k), ROCm reads long prompts 13-20% faster. Numbers: https://t.co/bojyBiXKH6 #Vulkan #ROCm #AMD #LocalLLM

[lejars.g @g_lejars]

@sudoingX RX 9070 XT 16GB on Linux (RADV 25.2.8): Vulkan out-generates ROCm 7.2.4 on Qwen3 14B Q4_K_M, 43.7 vs 35.5 tok/s at 30k ctx. ROCm reads long prompts 13-20% faster. Full bench: https://t.co/nvGstaVjwI #Radeon #llamacpp #Vulkan #LocalLLM

[lejars.g @g_lejars]

I ran my local AI assistant through 22 real scenarios, 5 runs each.

109/110 passed. The 1 miss: it counted 7 devices on my network where 5 were expected. Cause not verified yet. (3 more scenarios were skipped, not counted.)

Qwen3 14B on one RX 9070 XT, no cloud AI model. Measured, not claimed.

Method on my site 👇
https://t.co/top03Etq6f

#LocalLLM #Qwen3 #AIAgents #LLM #SelfHosted #AMD

[lejars.g @g_lejars]

How my local AI assistant picks a tool, in 3 layers 👇
1️⃣ Rules: reboot/shutdown run instantly
2️⃣ SetFit: tiny classifier, 1 of 38 tools in ms
3️⃣ qwen3:14b: only if SetFit confidence < 0.6
The big LLM is the last resort. 100% local 🔒
#LocalLLM #SetFit #Qwen #AIAgents

[lejars.g @g_lejars]

Same question. Two AIs.

A chatbot says "Yes, you're right!"
Mine says "No. The sky is blue."

Luna isn't a chatbot. She's my assistant, running on my own PC. 🌌

#LocalLLM #Qwen #AI #SelfHosted #AIAssistant #llamacpp

[lejars.g @g_lejars]

I asked Luna, my local AI assistant, to scan a folder for viruses. She loads ClamAV, finds the EICAR test file, then checks the firewall, open ports and pending updates. Qwen3 14B, running on my own PC. No cloud.
#LocalLLM #Qwen3 #selfhosted #Linux #cybersecurity #AI

[lejars.g @g_lejars]

Same GPU, same model, two backends. Vulkan (RADV) vs ROCm on an RX 9070 XT, Qwen3 14B Q4_K_M, llama.cpp b11177.

Generation: Vulkan is +9% faster at 1k context, up to +23% at 30k (43.7 vs 35.5 tok/s).
Prompt reading: Vulkan is 13–20% slower.
VRAM: about the same.

Median of 7 runs, fixed seed, 256 tokens.

#LocalLLM #Qwen3 #llamacpp #Vulkan #ROCm #AMD #RX9070XT #LLM

[lejars.g @g_lejars]

Qwen3 14B Q4_K_M on a Sapphire Pulse RX 9070 XT 16 GB. llama.cpp b11177, ROCm 7.2.4, flash attention, q8_0 KV cache. Median of 7 runs, 256 tokens generated:

1k ctx: 53.5 tok/s
4k: 50.2
8k: 47.3
16k: 42.2
30k: 35.5

Prompt processing: 2,152 tok/s at 1k, 1,242 at 30k.
f16 KV cache: only ~3-4% faster up to 8k, slower at 30k, and uses 2.3 GiB more VRAM.

One machine, desktop running, prompt cache off.

#LocalLLM #AMD #ROCm #llamacpp #Qwen3

[lejars.g @g_lejars]

@focalzero That's the point of the post: 16 GB is already the comfortable end. Below that you're limited to smaller models.

[lejars.g @g_lejars]

I tested Qwen3 14B's thinking mode on 8 questions (some tricky, some with typos), 3 runs each, same GPU.

No thinking: 18/24 correct, 4.1 s.
Thinking: 23/24, 11.5 s.
Temp 0.6 instead of 0.2: no gain.

It still invents names ("Haïmée" as a dwarf planet). Thinking helps with sloppy reading, not made-up facts.

#LocalLLM #Qwen3 #llamacpp #AMD #LocalAI

[lejars.g @g_lejars]

Local AI has a hardware access problem. 16 GB of VRAM is enough for a 14B model at 51.8 tok/s: 13.4 GiB used, about 2.5 GiB left.

Anything bigger means a GPU price that pushes it out of reach for most individuals. Models keep growing; the memory on mid-range cards doesn't.

#LocalLLM #AMD #ROCm #Qwen3 #LocalAI

[lejars.g @g_lejars]

Qwen3 14B (Q4_K_M) running 100% locally on a single AMD GPU with 16GB VRAM. 51.8 tokens/s measured live from llama-server: 440 tokens in 8.5 s. No cloud, no API. #LocalLLM #Qwen #llamacpp #AMD #SelfHosted #AI #LLM