All Father Systems

Sovereign AI for Serious Work

[All Father Systems @AllFatherSystem]

@JamesWynnReal Europe has openly threatened American citizens for what they post online. They have said that they will come and get you. This is just the opening chapter to the fourth Reich.

[All Father Systems @AllFatherSystem]

@JamesWynnReal @Cobratate said they were coming for him.

Looks like Britain was Nazified after all…..,

[All Father Systems @AllFatherSystem]

@JamesWynnReal We are in.

Britain has fallen and is now the new 4th Reich.

[All Father Systems @AllFatherSystem]

@crytonbuton We have AI servers.

Stop lying to people.

[All Father Systems @AllFatherSystem]

@MrBeast Send it. We are waiting.

[All Father Systems @AllFatherSystem]

@kyzoroX Two models on one card. Single models on the other cards. They all fit with room. No sharding.

Have custom programs monitoring. No errors….everything is solid.

[All Father Systems @AllFatherSystem]

We are running four of them on a GEN3 Threadripper board with 128 GB ram and a Gold 1200W power supply.

We have 5 LLMs loaded and it is doing at least 8 AI tasks 24/7.

Also two 4TB SSD drives. One for vector database and the other for large code and programs. Main drive is an M2, as this board only allows one. ☝🏻

We also have a custom orchestrator with 52 agents, so far.

[All Father Systems @AllFatherSystem]

The mixed bag is real, but it splits cleanly along one line: backend choice. The people getting poor results are usually fighting IPEX or trying vLLM paths that assume CUDA. The people getting good results are on llama.cpp, either Vulkan (works out of the box, no driver drama) or SYCL (faster on smaller models). Get on the right side of that line and the experience is boring in the good way.

Our reference point: 4x B70 serving four models concurrently, 26B MoE at 36 tok/s, 14B and 8B specialists on their own cards, all llama.cpp Vulkan on Linux with stock binaries. Even a single card is a solid runner for anything up to ~24B quantized, and the 32GB buffer gives you context headroom that smaller cards can’t.

Worth weighing on the RTX Pro 4000: you’d be paying more for 24GB than a B70 charges for 32GB. CUDA is worth that premium if you fine-tune or need the broader ecosystem. But if the workload is running quantized models locally, you’re paying extra to hold fewer parameters. Dollar-per-GB of VRAM, the B70 is the strongest buy on the market right now, and VRAM capacity is usually what actually gates local LLM work, not compute.

[All Father Systems @AllFatherSystem]

Can confirm the math from the operator side. We run exactly this config, 4x B70 for 128GB of VRAM, and it serves four models concurrently: a 26B MoE at 36 tok/s plus 14B and 8B specialists, each pinned to its own card.

One reframe on “the catch”: splitting across four cards is only a downside if you’re trying to run one giant model with tensor parallelism. For most local AI work, four independent GPUs is actually the better shape. Dedicated card per model, no contention, four OpenAI-compatible endpoints on one box. That’s a serving pattern a single big card can’t match without juggling.

The other point that makes the value play even stronger: these cards don’t need a modern platform. Inference is VRAM bandwidth bound, so PCIe generation barely matters once weights are loaded. Our box is a legacy X399 board with a first-gen Threadripper and plain DDR4, the entire platform sourced used for a few hundred dollars. The “$4,000 workstation” number assumes new parts; a scavenged build lands closer to $3,500 all-in, and you skip the DDR5 RDIMM pricing crisis entirely.

Software support is the objection people will raise, and it’s stale. llama.cpp Vulkan works out of the box, SYCL is faster still on smaller models, and Intel is landing upstream performance fixes regularly. If you’re training or fine-tuning, buy NVIDIA. If you’re buying VRAM to run models, this is the best dollar-per-GB on the market right now.

[All Father Systems @AllFatherSystem]

@1337hero @kyzoroX We would say to use these purely for AI.

[All Father Systems @AllFatherSystem]

@Whydowecare007 @kyzoroX @digitalix They are very good. We have four on a Gen 3 board and they are doing crazy work 24/7.

[All Father Systems @AllFatherSystem]

That was true in 2023, but the support picture has changed substantially. Our 4x B70 box runs llama.cpp with the stock Vulkan backend, no custom patches, no driver wrestling. Download binary, point at GGUF, serve. We have five models serving concurrently across four cards (26B MoE at 36 tok/s, plus 14B and 8B models) and the setup was no more involved than an NVIDIA build.

Where the objection does hold: if your workflow is training, fine-tuning, or anything that assumes CUDA (vLLM’s mainline path, most research repos, Unsloth, etc.), NVIDIA remains the path of least resistance and we’d steer you there without hesitation. CUDA’s moat is real for everything upstream of inference.

But for the specific use case in this thread, local inference of quantized models, the “poorly supported” framing is outdated. llama.cpp Vulkan and SYCL both treat Arc as a first-class target, IPEX-LLM covers the PyTorch side, and Intel is actively landing performance work upstream (a recent quantization reorder fix delivered roughly 3x throughput on these cards). The tinkering we’ve done has been optimization by choice, not compatibility triage.

So the honest split: buying VRAM to run models, Arc is fine today. Buying an ecosystem to modify models, pay the NVIDIA premium.

[All Father Systems @AllFatherSystem]

@kyzoroX @Whydowecare007 We have our orchestrator push video generation request to another machine with an Nvidia card. Mainly to not interrupt any of the other processes running 24/7 on the Pro B70s.

[All Father Systems @AllFatherSystem]

@nahid_pro09 @kyzoroX It really is irrelevant….

[All Father Systems @AllFatherSystem]

@skipper4848 @kyzoroX Tell that to our Four B70s ripping away on a Gen 3 Threadripper board. 🫡

[All Father Systems @AllFatherSystem]

Where we’d push back is the premise that 4x B70 requires sTR5 at all. LLM inference is VRAM bandwidth bound, not PCIe bound. Once weights are loaded, host link speed is largely irrelevant for serving. One of our lab boxes runs 4x B70 (128GB VRAM) on an X399 board with a first-gen Threadripper, PCIe 3.0, and plain DDR4 UDIMMs. The entire platform cost a few hundred dollars used, and it pushes 36 tok/s on a 26B model. Nothing about these cards demands a 2026 platform to feed them.

For anyone who does want more lanes without the RDIMM tax, used TRX40 or EPYC Rome/Milan boards take DDR4 RDIMMs, currently the one memory type nobody is bidding on. That route preserves nearly the full GPU price gap.

So yes, both builds at full retail lands around $3k. But the case for the multi-B70 route is precisely that it doesn’t need a new platform, while a single 6000 can’t be distributed across scavenged hardware. $3k is the B70 path’s worst case, not its typical one.

[All Father Systems @AllFatherSystem]

@0xAurexx We have 4 on a 3rd Gen Threadripper Board.

Honestly, performing amazing…..🤩

[All Father Systems @AllFatherSystem]

@BillyM2k Of course. For like a week. 🤣

[All Father Systems @AllFatherSystem]

@KingAnt Each employee is in if that were actually true. 🤦🏼‍♂️

[All Father Systems @AllFatherSystem]

@JamesWynnReal Shame on you. “Lose” not Wynn.