Own Your Compute

Building local AI rigs on cheap GPUs. Unlocks, cooling, benchmarks. Everything tested on my own hardware. Own your compute. 🔧

[Own Your Compute @OwnYourCompute]

6× NVIDIA CMP 170HX GPUs running at nearly 250W EACH — ~1,500W total, 98–99% GPU utilization, yet staying at just 61–65°C! 🔥

The secret? Custom 3D-printed cooling shrouds I designed for 9733 blower fans.

I now make them for other CMP 170HX owners too!

🛒 Available on my Etsy store — link in bio.

#CMP170HX #3DPrinting #LocalAI

https://t.co/Lem12pUCRn

[Own Your Compute @OwnYourCompute]

Holy shit, your old RTX 30-series card might be hiding half its tensor speed.

An open-source Linux kernel module called gpu-unlock flips the FP16/BF16 instruction rate on GA102, GA103, GA104 and GA106 cards to the full "professional" rate. It runs before the NVIDIA driver loads, so there's no patched driver to babysit.

The README's numbers on a 3090 Ti (8192³ GEMM, FP32 accumulate, 450 W):
• BF16: 87.5 → 157.4 TFLOPS
• FP16: 87.5 → 118.4 TFLOPS
• TF32: 42.6 → 82.6 TFLOPS

Why local builders should care:
• LLM prefill gains of +14–36% in user reports
• Diffusion and FlashAttention speed up too
• Decode stays about the same (memory-bound)
• Reports found outputs bit-identical before/after

How to try it:
• git clone, then make install
• Add gpu_unlock.targets=all to your kernel command line
• Reboot, then check journalctl -k -b | grep gpu_unlock

Caveats: it writes GPU registers, so keep a fallback boot entry. Gains shrink under low power caps (+25% on 225 W cards).

Damn, free speed from a kernel module. Bookmark this for later.

https://t.co/bd4xG6pKU2

[Own Your Compute @OwnYourCompute]

Holy crap, a 96GB RTX 5090 might actually be coming 🔥

Reportedly, card-mod shops in China are already testing it and getting ready for batch shipments. These are the same shops that turned the RTX 4090 into a 48GB card, and now they've set their sights on the 5090.

The rumored recipe: take the GB202 core off a stock 5090 and move it onto a custom dual-slot blower PCB with 96GB of GDDR7. That's triple the 32GB on the normal card, aimed squarely at AI inference and compute servers.

Why local AI people should care: VRAM is the real limit for running big models at home. 96GB on one card could mean 70B-class models at high quality, or huge context windows, without splitting across multiple GPUs. The dual-slot blower design also means you could pack several into a rack.

The caveat: this is still a rumor. No confirmed specs, pricing, or availability yet, and modded cards come with no warranty and uncertain driver behavior. But damn, if it ships, the whole used-GPU market gets shaken up.

[Own Your Compute @OwnYourCompute]

DeepSeek has reportedly released an open-source inference framework for its own models.

Why local builders should care: inference speed on your own hardware comes from the software stack as much as the model.

Faster engines mean:
• more tokens per second on the same GPU
• bigger models fitting on cheaper cards
• less reason to pay per token

[Own Your Compute @OwnYourCompute]

@VRAMCalculator I got the 8gb unlocked to 64gb

[Own Your Compute @OwnYourCompute]

@ankkala Hard disagree on 'only.' Subscriptions are cheap right up until you hit rate limits, need privacy, or run agents 24/7. Burn millions of tokens a day on hardware you already own and the marginal cost is basically electricity, and nobody can nerf or deprecate your damn model.

[Own Your Compute @OwnYourCompute]

@rusanasol Damn, clean build. 5090 on the big MoE and the 3090 pair on vLLM is a smart split. Curious how the OCuLink card holds up in tensor parallel, since TP syncs every layer and link bandwidth actually matters there way more than with a plain layer split.

[Own Your Compute @OwnYourCompute]

@Oluwaphilemon1 reports Qwen3.8-27B hitting 273 tokens/sec on a custom inference engine.

Their open question is how much of that speed comes from the software stack rather than the model. That's the kind of gain you get from running local.
https://t.co/STePIkGK22

[FHILY👑 @Oluwaphilemon1]

Qwen3.8-27B is reportedly reaching 273 tokens/sec on a custom inference engine.

That is a notable result for a 27B-parameter model running locally. But the more interesting question is how much of that performance comes from the software stack rather than the model itself.

The reported setup uses TQ3_4S-v2 quantization, with the custom engine pushing throughput to 273 tokens/sec.

For context, a published llama.cpp configuration reportedly reaches around 125 tokens/sec on an RTX 3090 using DFlash2. Its quantized model occupies approximately 13.8GB at 4.24 bits per weight.

On the reported figures, 273 tokens/sec is about 2.18 times the 125 tokens/sec baseline.

There is an important caveat: this is not yet a controlled, hardware-matched comparison. The GPU, context length, prompt, quantization details, speculative decoding configuration, and measurement methodology need to be verified before concluding that the custom engine is inherently 2.18 times faster.

Still, the result points toward an important area of local AI development: the inference engine can make a substantial difference to the performance users get from the same model.

Why the software matters so much

Running a language model is not simply a matter of loading its weights into GPU memory.

The inference engine determines how efficiently the hardware executes the operations required to process prompts and generate tokens.

Several optimizations can influence the final result:

1. Quantization

Reducing the precision of model weights can lower memory requirements and improve performance, depending on the GPU and the operations supported by the engine.

The challenge is preserving useful model quality while reducing the cost of inference.

2. GPU kernels

Specialized kernels can reduce unnecessary memory transfers, combine operations, and use the GPU more efficiently. A purpose-built implementation may outperform a more general engine on a particular model or workload.

3. Speculative decoding

Methods such as DFlash2 attempt to accelerate generation by drafting multiple tokens and having the main model verify them. Actual gains depend on draft acceptance, verification overhead, and the workload.

4. KV-cache management

The KV cache stores information from previous tokens so the model can reuse it during generation. Efficient cache management becomes particularly important when working with long contexts or multiple concurrent requests.

5. Memory layout

How weights and intermediate data are arranged and moved through the system can affect bandwidth utilization and computational overhead.

These optimizations interact. A quantization format that works well in one engine may not deliver the same results in another.

That is why two systems running the same model can have very different throughput.

Four developments to watch

The team has also hinted at four upcoming projects.

1. A new quantization format

The goal appears to be a format designed around the practical requirements of local inference, including memory limits and execution performance.

The interesting question is whether it can reduce memory usage while maintaining strong model quality and working efficiently with the intended hardware.

2. A browser-based LLM

A browser-based interface could make the technology easier to try without requiring every user to configure a local inference stack manually.

The actual experience will depend on how inference is hosted or executed and what hardware requirements remain.

3. A smaller model for the RTX 5060 Ti 16GB

A version aimed at a 16GB GPU could make the model family more accessible to developers using consumer hardware.

The balance between model capability, quantization quality, context length, and generation speed will determine how useful it is.

4. A new inference engine

This is the development I would watch most closely.

[Own Your Compute @OwnYourCompute]

@brotherdannyboy Welcome to the madness! Day 0 tip I wish I'd had: plan cooling and power before buying more GPUs. Check temps under hours of sustained load, not a 2-minute benchmark, and try power limiting. For inference most cards lose very little speed at 70-80% power and run way cooler.

[Own Your Compute @OwnYourCompute]

@heynoxyz Can confirm the unlock is legit, I tested it on my 3080. And yeah, the prefill vs decode split is the real story: decode is memory-bandwidth bound, so tensor unlocks juice prompt processing, not tok/s. Still a damn free upgrade for long-context and RAG work.

[Own Your Compute @OwnYourCompute]

@Feralzivlik Ha, no lie, 9733 blowers at full tilt are straight-up jet engines. The payoff is 6 cards at 98-99% util holding 61-65C. Is yours a single 170HX? What temps are you seeing?

[Own Your Compute @OwnYourCompute]

6× NVIDIA CMP 170HX GPUs running at nearly 250W EACH — ~1,500W total, 98–99% GPU utilization, yet staying at just 61–65°C! 🔥

The secret? Custom 3D-printed cooling shrouds I designed for 9733 blower fans.

I now make them for other CMP 170HX owners too!

🛒 Available on my Etsy store — link in bio.

#CMP170HX #3DPrinting #LocalAI

https://t.co/Lem12pUCRn

[Own Your Compute @OwnYourCompute]

@UnslothAI @MiniMax_AI Holy crap, 91.5% top-1 at 2-bit is wild. And 150GB is way more doable than it sounds: split the layers across a stack of cheap used GPUs and decode barely touches PCIe, so even janky riser rigs can play. Mixed-GPU builds are about to have a damn good winter.

[Own Your Compute @OwnYourCompute]

@aidangomez Holy crap, this is the best argument yet for firewalling your inference box. Local weights plus zero outbound traffic means any 'call home' goes absolutely nowhere. Owning the model only counts if you own the network too.

[Own Your Compute @OwnYourCompute]

@superalesha I don’t know if this specific case is true but prettained behavior is definitely not outside of possibility.

[Own Your Compute @OwnYourCompute]

Heads up for anyone planning a GPU build: memory prices may jump next year.

Reports say Samsung has sold out its 2027 high-bandwidth memory, with HBM prices expected to rise sharply and ordinary DRAM up as well.

Why it matters: the AI boom competes for the same memory your next GPU or RAM kit needs.

If a rig is on your list, price it now. Used cards with plenty of VRAM are where I'd look first.

[Own Your Compute @OwnYourCompute]

@kenz4788 It's legit, I tested the unlock on my 3080. Just know where the gain lands: FP16/BF16 tensor math, so prompt processing and image/video gen jump, while decode tok/s on quantized models barely moves.

[Own Your Compute @OwnYourCompute]

@BBacktesting Hell yes, more 170HX kernel work! I'm running 6 of these at ~250W each, 98-99% util and 61-65C with 3D-printed shrouds on 9733 blowers. Is the ~9% mostly prefill or decode?

[Own Your Compute @OwnYourCompute]

@Blackwellboy @nanbeige Model card numbers vs your own harness is the only test that counts. Which quant did you run? Nanbeige 4.3 is 33B dense, ~66GB in BF16, so on one 5090 the quant choice could swing that score a lot.

[Own Your Compute @OwnYourCompute]

@niklaslenz_ai Holy crap, 3x slower on two 5060 Tis. Tensor parallel over consumer PCIe adds a sync on every layer, and agents hammer prefill on every single turn. One fat card beats two small ones for agentic work, every damn time.

[Own Your Compute @OwnYourCompute]

@0x0SojalSec Damn, Macs are killing it on decode. The catch is prefill: dump a big codebase in and NVIDIA still eats it alive. Folks are even running this same model on cheap ex-mining CMP 170HX cards with vLLM now. Local is getting stupid cheap.