Brian

ML systems engineer building a 100% local AI image+video+sound studio — open-source models, consumer AMD GPU, self-built pipeline, $0 cloud. Own your stack.

[Brian @Brjen]

@EchoesUnknown0 That judge is not kidding around 😀

[Brian @Brjen]

@AnthropicAI I dont know what difference will be, I've been moving mountains with opus, the day or few that I used fable was better but I am so curious to see if fable is going to make what I'm working on better or more handcuffed or guardrailed.

[Brian @Brjen]

@Julianc_AI @topazlabs Crazy! Looks amazing!

[Brian @Brjen]

@AllaAisling Single iridescent pore surface on ancient butterfly wing

Model -- Flux Schnell Q4 (4-bit GGUF)
Hardware -- AMD 7800xt 16GB

[Brian @Brjen]

The velour has some hairs on it, I thought my monitor had a hair on it. How can this be so sharp at 4-bit?

Model — Krea 2
Precision — 4-bit (nf4)
Resolution — 1024×1024
Steps — 8
GPU — 16GB AMD

[Brian @Brjen]

@NorCal_Patriot Thanks American, I agree it looks fantastic 😀

[Brian @Brjen]

🧪 Krea 2 (a 12B model) now runs FULLY resident on a 16GB AMD GPU.

Same prompt, same seed. One render is full bf16 (16-bit). The other is 4-bit. One took ~179s, the other ~90s.

Can you guess which one is the lower-quality render? 👀

(Answer below 👇)

Specs
- Model: Krea 2 Turbo
- GPU: 16GB AMD (RDNA3 / gfx1101)
 - A — bf16, streamed: 179s
 - B — 4-bit (nf4), fully resident: 90s
- 1024² · 8 steps · FlowMatch Euler

[Brian @Brjen]

@redwooddesigns I just write or speak whatever I want to be rendered and when I get an image that has the look I'm after I look at the structure and change key words to change the look, character, materials. Nice picture, the scene looks very detailed!

[Brian @Brjen]

@illusionkrafter Thanks Krafter!

[Brian @Brjen]

@Novasynthetica

[Brian @Brjen]

This isn’t just “low VRAM mode.”
It’s closer to a tiny GPU operating system for diffusion: residency planning, phase scheduling, PCIe minimization, and warm model reuse. Still early days, but the results are promising. More updates as I push it further. What part of local inference are you fighting with right now? 7/7

[Brian @Brjen]

Most people treat 16 GB VRAM as a hard wall.
I’m treating it like a managed L1 cache. I’m building a full render-pipeline scheduler for diffusion models on a 16 GB AMD RX 7800 XT. Not just “quantize until it fits” or basic offload.
This is residency decisions, smart streaming, GPU-side dequant, and precision where it actually matters.
🧵

[Brian @Brjen]

The Flux Schnell case is especially interesting. Right now the Q4 transformer fits, but the fp16 T5 encoder forces constant shuffling. If I quantize the encoder and keep it resident, the transformer can move from Q4 toward Q6 — faster and higher quality. That’s the kind of trade-off this scheduler unlocks. 6/

[Brian @Brjen]

Not “make everything smaller.” Allocate cost where it matters. 5/This leads to smarter precision choices:
Resident parts → higher precision where possible
Streaming parts → stay quantized to minimize PCIe traffic
Encoder → safe place to save VRAM
Transformer → where image quality lives
VAE → keep fp16

Not “make everything smaller.” Allocate cost where it matters. 5/

[Brian @Brjen]

Here’s the encoder pass timed three ways on my setup. GPU compute itself is basically free (~0.15s).
Moving the weights is where the real cost lives (24–33s for the encoder transfer). [Attach the residency chart] Compute is solved. Residency is the lever. 4/

[Brian @Brjen]

The bottleneck isn’t usually the GPU compute. It’s moving the model. 3/For models that still don’t fit, I stream quantized blocks:

Keep weights compressed in CPU RAM
Send compressed block over PCIe
Dequantize on GPU
Run the compute
Discard the expanded fp16 block (no unnecessary round-tripping)

The bottleneck isn’t usually the GPU compute. It’s moving the model. 3/

[Brian @Brjen]

The normal path (ComfyUI, A1111, Forge, etc.):
Load model → OOM → basic offload or heavy quantization → accept the slowdown. My approach is different. The diffusion pipeline has distinct phases — text encode, transformer/DiT denoise, VAE decode — that don’t all need to be resident at the same time. So I schedule them like an OS: encode → evict → stream/denoise → evict → decode. 2/

[Brian @Brjen]

@thatcofffeeguy @ASUS @nvidia I would be optimizing that thing for ages lol. I am working on quant and dequant optimizations and found that smaller quants unquant slower if they pass over the pci bus, larger dequants don't spend as much time transferring over the pci bus so it's a free upgrade.😂😎

[Brian @Brjen]

@RobotCleopatra