Own Your Compute

Building local AI rigs on cheap GPUs. Unlocks, cooling, benchmarks. Everything tested on my own hardware. Own your compute. 🔧

[Own Your Compute @OwnYourCompute]

6× NVIDIA CMP 170HX GPUs running at nearly 250W EACH — ~1,500W total, 98–99% GPU utilization, yet staying at just 61–65°C! 🔥

The secret? Custom 3D-printed cooling shrouds I designed for 9733 blower fans.

I now make them for other CMP 170HX owners too!

🛒 Available on my Etsy store — link in bio.

#CMP170HX #3DPrinting #LocalAI

https://t.co/Lem12pUCRn

[Own Your Compute @OwnYourCompute]

Real talk, local AI builders. You've got ~$1,000 for GPUs. What are you buying?

A) One used 24GB card
B) Two used 16GB cards
C) Four or more cheap 8-12GB cards
D) Something weird (unlocked mining cards, old server GPUs)

Why this is a damn hard call:
• One big card = simplest setup, no model splitting, usually the lowest power draw
• Multiple cards = more total VRAM per dollar, but you need slots, a serious PSU, cooling and enough PCIe lanes
• llama.cpp and vLLM can split a model across cards, so pooled VRAM works, but one card is always less hassle

Drop your pick and the model you're trying to run. I want to see who's got the wildest answer. 👇

[Own Your Compute @OwnYourCompute]

This is absolutely nuts: a 27B model on ONE 24GB RTX 3090, served with vLLM, 150k context, OpenAI-compatible API.

HyperQwen is an open-source (Apache 2.0) patch series on a pinned vLLM plus a model-prep pipeline for Qwen3.8-27B. The README's numbers on a 3090 at 250 W:
• 127 tok/s single-stream
• ~1,035 tok/s aggregate at 64 concurrent requests
• 381 tok/s when the answer quotes its own prompt

Where it shines:
• Coding assistants applying edits
• RAG front-ends and document rewrites
• A private API backend for your own agents

Try it:
• git clone the repo, cp .env.example .env
• docker compose --profile single up -d (or --profile batch for many users)
• Set a VLLM_API_KEY before exposing the port

Caveats: the 381 figure only happens when output repeats the input. Normal chat is the 127 mode. First start pulls a 9.5 GB image and requantizes ~20 GB of weights. The README says it has only been run on 3090 and 3080 cards so far.

Holy crap, software alone. Bookmark this for later.

https://t.co/TdCtfl4e1o

[Own Your Compute @OwnYourCompute]

@AnandButani @Apple Finally someone said it. Decode speed is memory bandwidth divided by model size, period. Same reason tensor unlocks on RTX 30 cards juice prefill but barely touch tok/s. Want fast local generation? Shop GB/s per dollar, not TOPS.

[Own Your Compute @OwnYourCompute]

@mervenoyann Hell yes, and zero excuses goes for hardware too. I've got 6 used CMP 170HX mining cards pegged at 98-99% util serving open models 24/7. Once you own the box, the marginal cost per token is basically electricity.

[Own Your Compute @OwnYourCompute]

@GameGPU_com Holy crap, 40-series owners get the unlock too! Know where the gain lands though: I tested the 30-series unlock on my 3080, and it's FP16/BF16 tensor math, so prefill, training and image/video gen jump while decode tok/s barely moves. Decode is bandwidth bound.

[Own Your Compute @OwnYourCompute]

Holy crap, Google just dropped Genie 4 and it's a real-time interactive world model 🌐

You type a prompt and you're walking around inside a world. A WASD keyboard overlay shows you steering it live, and you can keep typing to change the scene on the fly: add a shooting star, drop a piano in the snow, or have giant alien tentacles tear up the city behind a moped. It generates ambient sound and character speech too, not just pixels.

The part that matters for us: they showed 16 simulated robot arms pipetting in a grid, then a real arm doing the same task in a lab. Practicing robots in a generated world is the real endgame here.

Their own disclaimer says the clips are for illustrative purposes and shortened, and there are no resolution or frame rate numbers, so I'm waiting to see how it runs outside a demo.

[Own Your Compute @OwnYourCompute]

Stop guessing if a model fits your GPU. The napkin math takes 10 seconds.

Weights ≈ parameters × bits per weight ÷ 8

• 8B at 4-bit ≈ 4GB
• 32B at 4-bit ≈ 16GB
• 70B at 4-bit ≈ 35GB
• 70B at 8-bit ≈ 70GB
• 70B at 16-bit ≈ 140GB

Damn, that's why 16-bit 70B is a data-center problem and 4-bit is a home-lab problem.

But weights are only part of the bill:
• KV cache grows with context length, so long context eats several more GB
• CUDA and runtime overhead take a chunk
• Real Q4_K_M files come out a bit bigger than the raw 4-bit math because some tensors stay at higher precision

Safe rule of thumb: take the file size, add 10-20% headroom, then add KV cache for the context you actually use.

That's why a 70B at 4-bit won't fit on one 24GB card, but two 24GB cards give you 48GB to split it across.

Caveat: MoE models only activate some experts per token, but you still need all the weights in memory (or offloaded).

Do the math before you buy the card.

[Own Your Compute @OwnYourCompute]

Holy crap, someone made ComfyUI wires look like an actual professional cable bundle and I can't stop watching this.

You know that spaghetti mess of wires in a big ComfyUI graph? This extension zip ties them into one clean ribbon. Hold Z, drag a line across the wires, and boom, seven wires get bundled into a tidy braid with little zip ties. Then you drag the ties around and the whole bundle bends with them. This is insane for a graph that looks like a plate of noodles.

What it does, straight from the README:
- Hold Z and drag across wires to tie them together
- Drag a tie to move it, Shift-drag to rotate it in 15 degree steps
- Right-click a tie to change colour or remove it
- Ties get a random cable-tie colour, or you can set them to match the wires
- Ties are saved in the workflow itself, so they save, load and undo with the graph
- No new nodes and no Python dependencies, it's one frontend file

How to try it:
comfy node install comfyui-zip-ties

Or search for Zip Ties in ComfyUI-Manager. Manual install:
cd ComfyUI/custom_nodes
git clone https://t.co/LvVtSvKEXY

Honest caveat: wires that pass through native reroutes can't be tied, and the README says it was tested on ComfyUI 0.42.0 with frontend 1.57.1. The creator calls it just for fun, but damn, it's a satisfying one. MIT licensed.

#ComfyUI

Bookmark this for later.
safzanpirani/comfyui-zip-ties

[Own Your Compute @OwnYourCompute]

PCIe x1 is supposed to kill LLM inference. I ran a 262K-token context across two CMP 170HX cards on PCIe Gen2 x1 and it didn't.

The setup:
• 2× unlocked CMP 170HX
• Both GPUs on PCIe Gen2 x1 links
• Qwen3.8-Flash-Next-W4A16-FP8PLE in vLLM 0.29.0, pipeline parallel 2, MTP 3

The numbers:
• 186.93 output tokens/sec at 8 concurrent requests
• 4,770 input tokens/sec on a cold 262K prefill
• 121,035 effective input tokens/sec with a warm cached prefix
• Prefix caching cut time to first token by up to 25x
• Tool calling and long-context retrieval passed
• No OOMs, CUDA errors, NCCL errors or deadlocks

Why it works: once the model sits in VRAM, the slow link barely matters, because the GPUs aren't constantly shuffling weights over it. Cheap cards can do real serving.

How far do you think this scales?

[Own Your Compute @OwnYourCompute]

Holy crap, this fixes the thing every agent-plus-notes setup gets wrong: finding the right file.

QMD is a free, MIT-licensed, on-device search engine for your markdown notes, meeting transcripts, docs and knowledge bases. It's built to be called by agents.

What the README says:
• Hybrid search: BM25 keywords + vector semantic search + LLM re-ranking
• Everything runs locally through node-llama-cpp with GGUF models
• Default models are tiny: ~300MB embeddings, ~640MB reranker, ~1.1GB query expansion
• Has --json output and an MCP server, so Claude Code or any MCP client can query your notes
• You can attach context to each collection so results make more sense to the LLM

How to try it:
npm install -g @tobilu/qmd
qmd collection add ~/notes --name notes
qmd embed
qmd query "how did I fix that CUDA error"

Caveat: first run downloads the GGUF models from Hugging Face, and if you swap the embedding model you have to re-run qmd embed.

Your local rig can quietly run your whole second brain. Bookmark this for later.

tobi/qmd

[Own Your Compute @OwnYourCompute]

Holy shit, your old RTX 30-series card might be hiding half its tensor speed.

An open-source Linux kernel module called gpu-unlock flips the FP16/BF16 instruction rate on GA102, GA103, GA104 and GA106 cards to the full "professional" rate. It runs before the NVIDIA driver loads, so there's no patched driver to babysit.

The README's numbers on a 3090 Ti (8192³ GEMM, FP32 accumulate, 450 W):
• BF16: 87.5 → 157.4 TFLOPS
• FP16: 87.5 → 118.4 TFLOPS
• TF32: 42.6 → 82.6 TFLOPS

Why local builders should care:
• LLM prefill gains of +14–36% in user reports
• Diffusion and FlashAttention speed up too
• Decode stays about the same (memory-bound)
• Reports found outputs bit-identical before/after

How to try it:
• git clone, then make install
• Add gpu_unlock.targets=all to your kernel command line
• Reboot, then check journalctl -k -b | grep gpu_unlock

Caveats: it writes GPU registers, so keep a fallback boot entry. Gains shrink under low power caps (+25% on 225 W cards).

Damn, free speed from a kernel module. Bookmark this for later.

https://t.co/bd4xG6pKU2

[Own Your Compute @OwnYourCompute]

Holy crap, a 96GB RTX 5090 might actually be coming 🔥

Reportedly, card-mod shops in China are already testing it and getting ready for batch shipments. These are the same shops that turned the RTX 4090 into a 48GB card, and now they've set their sights on the 5090.

The rumored recipe: take the GB202 core off a stock 5090 and move it onto a custom dual-slot blower PCB with 96GB of GDDR7. That's triple the 32GB on the normal card, aimed squarely at AI inference and compute servers.

Why local AI people should care: VRAM is the real limit for running big models at home. 96GB on one card could mean 70B-class models at high quality, or huge context windows, without splitting across multiple GPUs. The dual-slot blower design also means you could pack several into a rack.

The caveat: this is still a rumor. No confirmed specs, pricing, or availability yet, and modded cards come with no warranty and uncertain driver behavior. But damn, if it ships, the whole used-GPU market gets shaken up.

[Own Your Compute @OwnYourCompute]

DeepSeek has reportedly released an open-source inference framework for its own models.

Why local builders should care: inference speed on your own hardware comes from the software stack as much as the model.

Faster engines mean:
• more tokens per second on the same GPU
• bigger models fitting on cheaper cards
• less reason to pay per token

[Own Your Compute @OwnYourCompute]

@VRAMCalculator I got the 8gb unlocked to 64gb

[Own Your Compute @OwnYourCompute]

@ankkala Hard disagree on 'only.' Subscriptions are cheap right up until you hit rate limits, need privacy, or run agents 24/7. Burn millions of tokens a day on hardware you already own and the marginal cost is basically electricity, and nobody can nerf or deprecate your damn model.

[Own Your Compute @OwnYourCompute]

@rusanasol Damn, clean build. 5090 on the big MoE and the 3090 pair on vLLM is a smart split. Curious how the OCuLink card holds up in tensor parallel, since TP syncs every layer and link bandwidth actually matters there way more than with a plain layer split.

[Own Your Compute @OwnYourCompute]

@Oluwaphilemon1 reports Qwen3.8-27B hitting 273 tokens/sec on a custom inference engine.

Their open question is how much of that speed comes from the software stack rather than the model. That's the kind of gain you get from running local.
https://t.co/STePIkGK22

[FHILY👑 @Oluwaphilemon1]

Qwen3.8-27B is reportedly reaching 273 tokens/sec on a custom inference engine.

That is a notable result for a 27B-parameter model running locally. But the more interesting question is how much of that performance comes from the software stack rather than the model itself.

The reported setup uses TQ3_4S-v2 quantization, with the custom engine pushing throughput to 273 tokens/sec.

For context, a published llama.cpp configuration reportedly reaches around 125 tokens/sec on an RTX 3090 using DFlash2. Its quantized model occupies approximately 13.8GB at 4.24 bits per weight.

On the reported figures, 273 tokens/sec is about 2.18 times the 125 tokens/sec baseline.

There is an important caveat: this is not yet a controlled, hardware-matched comparison. The GPU, context length, prompt, quantization details, speculative decoding configuration, and measurement methodology need to be verified before concluding that the custom engine is inherently 2.18 times faster.

Still, the result points toward an important area of local AI development: the inference engine can make a substantial difference to the performance users get from the same model.

Why the software matters so much

Running a language model is not simply a matter of loading its weights into GPU memory.

The inference engine determines how efficiently the hardware executes the operations required to process prompts and generate tokens.

Several optimizations can influence the final result:

1. Quantization

Reducing the precision of model weights can lower memory requirements and improve performance, depending on the GPU and the operations supported by the engine.

The challenge is preserving useful model quality while reducing the cost of inference.

2. GPU kernels

Specialized kernels can reduce unnecessary memory transfers, combine operations, and use the GPU more efficiently. A purpose-built implementation may outperform a more general engine on a particular model or workload.

3. Speculative decoding

Methods such as DFlash2 attempt to accelerate generation by drafting multiple tokens and having the main model verify them. Actual gains depend on draft acceptance, verification overhead, and the workload.

4. KV-cache management

The KV cache stores information from previous tokens so the model can reuse it during generation. Efficient cache management becomes particularly important when working with long contexts or multiple concurrent requests.

5. Memory layout

How weights and intermediate data are arranged and moved through the system can affect bandwidth utilization and computational overhead.

These optimizations interact. A quantization format that works well in one engine may not deliver the same results in another.

That is why two systems running the same model can have very different throughput.

Four developments to watch

The team has also hinted at four upcoming projects.

1. A new quantization format

The goal appears to be a format designed around the practical requirements of local inference, including memory limits and execution performance.

The interesting question is whether it can reduce memory usage while maintaining strong model quality and working efficiently with the intended hardware.

2. A browser-based LLM

A browser-based interface could make the technology easier to try without requiring every user to configure a local inference stack manually.

The actual experience will depend on how inference is hosted or executed and what hardware requirements remain.

3. A smaller model for the RTX 5060 Ti 16GB

A version aimed at a 16GB GPU could make the model family more accessible to developers using consumer hardware.

The balance between model capability, quantization quality, context length, and generation speed will determine how useful it is.

4. A new inference engine

This is the development I would watch most closely.

[Own Your Compute @OwnYourCompute]

@brotherdannyboy Welcome to the madness! Day 0 tip I wish I'd had: plan cooling and power before buying more GPUs. Check temps under hours of sustained load, not a 2-minute benchmark, and try power limiting. For inference most cards lose very little speed at 70-80% power and run way cooler.

[Own Your Compute @OwnYourCompute]

@heynoxyz Can confirm the unlock is legit, I tested it on my 3080. And yeah, the prefill vs decode split is the real story: decode is memory-bandwidth bound, so tensor unlocks juice prompt processing, not tok/s. Still a damn free upgrade for long-context and RAG work.