People tend to assume that owning a genuinely capable, fast local AI requires a $4,000 GPU or a maxed-out Mac Studio.
Over the past couple of days, I realized it really doesn't have to be that expensive — or that intimidating. Everyday people can actually build and own one.
Sitting on my desk right now is a $220 (1,568 CNY) pile of secondhand server parts running a 125B sparse MoE model — Qwen3.8-Flash-Next (IQ2_XS) — completely offline on my home LAN at ~55 tok/s sustained (up to 83.8 tok/s on code) with a full 256K context window. No subscriptions, no rate limits, no data leaving my room.
The entire $220 build:
- Motherboard + CPU + RAM: Used Huananzhi X99-BD3 (quad-channel) + Xeon E5-2686 v4 + 64GB DDR3 ECC = $102
- GPU: Used AMD Instinct MI50 16GB HBM2 (zero Tensor Cores) = $106
- Cooler + 700W PSU (used) = $14
- Storage: Old 256GB SSD from a drawer = $0
- Case: Open desk + free winter air = $0
What surprised me most wasn't just what old hardware can still do — it's how much the technical barrier has collapsed.
You don't need to be a hardware veteran or a GPU kernel engineer anymore. I used Gemini Web to help me pick compatible used parts, wire up the board, and install Linux. Then I handed SSH access to Gemini 4 Carbon in Antigravity, nudging its direction here and there over ~24 hours while it turned "retired e-waste" into something wild:
- It dumped and binary-patched the GPU's PowerPlay table to unlock 1.13 TB/s of HBM2 bandwidth without a hardware flasher, and partitioned the Xeon's L3 cache so PCIe traffic stopped stalling CPU cores.
- It unpacked and disassembled the compiled GPU binaries on its own, caught a hidden ROCm compiler bug that was spilling 552 registers to VRAM, wrote a custom <32KB INT8 FlashAttention kernel (doubling prefill to 685 tok/s), and even hand-wrote raw AMD GCN assembly to stream activations straight from HBM2 into shared memory.
- When 256K context squeezed VRAM, we moved cold KV cache into pinned DDR3 RAM and kept only a hot 16K window in HBM2 — keeping ~55 tok/s rock-solid all the way to 256K tokens.
There's something quietly magical about opening a browser tab on your phone, chatting at 55+ tok/s with a 125B model sitting on your own desk, and knowing every weight and token belongs to you — for the price of a pair of earbuds.
If you've ever wanted a real AI box of your own, don't let flagship GPU prices scare you off. Grab some forgotten server silicon, pair up with a coding agent, and tinker.
Full notes, configs & open-source fork if anyone wants to try:
https://t.co/f3QT3IS60y
GPU power (hwmon): ~28W idle, ~100–130W during decode, and ~160–190W under sustained 256K prefill (~210–250W whole-rig decode / ~320–350W peak at the wall).
Haven't benchmarked IQ3_S yet — it needs ~55GB for weights alone (tight on 64GB RAM with 256K KV cache, plus my free 256GB SSD is full lol), though the fused Wave64 IQ3_S kernel is already wired up in the fork!
I gave @GoogleAI 's Gemini 4 Carbon (via Antigravity) root access to a $220 pile of secondhand server parts for 24 hours straight.
When I came back, it was running a 125B MoE model — Qwen3.8-Flash-Next (IQ2_XS) — locally at ~55 tok/s sustained decode (83.8 tok/s code peak) and 615–685 tok/s prefill, holding ~55 tok/s all the way to 256K context on a GPU with ZERO Tensor Cores.
Repo & benchmarks: https://t.co/f3QT3IS60y
The 1,568 CNY (~$220 USD) build sheet:
- Motherboard + CPU + RAM: Huananzhi X99-BD3 (quad-channel memory rivaling DDR4 bandwidth) + Xeon E5-2686 v4 (18C/36T) + 64GB DDR3-1866 ECC = $102 (718 CNY)
- Cooler: Used tower cooler = $4 (30 CNY)
- GPU: Used AMD Instinct MI50 16GB HBM2 (Vega 20, no matrix cores) = $106 (750 CNY)
- PSU: Used 700W = $10 (70 CNY)
- Storage: Spare 256GB SSD from a drawer = $0
- Case: Open-air desk + free winter weather cooling = $0
Gemini Web helped me pick the parts, assemble the rig, and install Linux. Then I handed the whole machine over to Gemini 4 Carbon in Antigravity and told it to hold nothing back. Without a doubt, Gemini 4 Carbon's relentless exploratory drive — paired with the deep world knowledge and lateral thinking the Gemini series has always been known for — made all the difference. Here is what it pulled off over 24 hours on its own:
1. Squeezed hidden platform bandwidth (zero hardware flashing)
Unlocked PCIe 4096B DMA (13.3 GB/s), locked Xeon Uncore clocks, isolated background PCIe traffic via Intel L3 CAT (16x less jitter), and injected a runtime Linux PowerPlay table pushing the MI50 to 1700 MHz / 1.13 TB/s HBM2.
2. Rebuilt the compute stack from scratch — down to bare-metal GCN assembly
Because Vega 20 has zero Tensor Cores, Gemini went straight after the silicon limits:
- Custom Wave64 & AVX2 kernels: Rewrote fused GPU MoE kernels with bank-conflict-free shared-memory bitplane transposes, 3.05x faster expert routing, and 7.84x faster 248K-vocab argmax at 0 VGPR spills across the entire engine — plus register-blocked AVX2 kernels speeding up 17 Xeon CPU cores by 1.73x.
- Custom <32KB INT8 FlashAttention: Hand-crafted an online INT8 dot-product (v_dot4_i32_i8) FlashAttention kernel strictly under 32 KB shared memory to double GPU occupancy per CU, paired with a 112-shape rocBLAS tuning table — nearly doubling prefill to 685 tok/s.
- Raw AMD GCN inline assembly: When the HIP compiler still left performance on the table, it dropped to bare assembly — using hardware async DMA (buffer_load_dword ... lds) to stream activations straight from HBM2 into on-chip shared memory without touching registers, and replacing shared-memory shuffles across 7 core GPU modules with 1-cycle cross-lane VALU DPP instructions. Verified 100% bit-identical on greedy decode.
3. Kept ~55 tok/s all the way to 256K context + 83.8 tok/s tree speculation
Fixed 263 MiB of draft-ring waste and streamed cold 256K KV blocks from 2.4 GiB of pinned DDR3 while keeping a hot 16K window in HBM2 (>90% hit rate), reclaiming ~2,000 VRAM expert slots for zero speed drop at 256K — topped off with a high-priority draft stream and 2-branch tree speculation hitting 83.8 tok/s on code.
Turns out you don't need $5,000 of silicon when your AI agent is willing to write assembly at 3 AM — as long as winter doesn't end and melt my $0 case.
Next up: challenging DeepSeek V4.1 Flash (~700B) on a $1,000 budget?
Big thanks to @coldniko for the original Strata work that made this possible.
Also big shoutout to @OfficialLoganK@joshwoodward@LyalinDotCom — Gemini 4 Carbon in Antigravity went full sicko mode on this one.
Both mattered, just on different axes!
Quantization-side tweaks won the baseline war: we kept weights at IQ2_XS, but rewriting how IQ2_XS / Q8_1 dequantize & dot-product without Tensor Cores (branchless INT8 dot-products, shared-memory bitplane transposes, direct HBM2-to-LDS DMA, FP16 prompt GEMMs) — plus switching 256K KV cache to k8v4 with only a 16K hot window in VRAM so 2,000+ extra experts fit in HBM2 — gave the universal lift (+52% decode, ~2x prefill, zero drop at 256K).
Since this is single-stream (parallel=1), "batching" came down to speculative verification width & MoE tile grouping: tuning --spec 4 (spec_min_p=0.65), isolating the drafter on a high-priority HIP stream, enabling I=128 sorted MoE prefill tiles, and adding 2-branch tree verification (--spec-tree 3) pushed code bursts from the ~55 tok/s floor up to 83.8 tok/s.
Just pulled the exact token logs from Antigravity across the whole ~24h run (~1,613 agent turns total):
- Total input: 246.3M tokens — 228.3M cached (92.7% cache hit rate!) + 18.0M uncached
- Total output (code + reasoning): 2.03M tokens
It ran under my Antigravity plan quota, though with that 92.7% cache hit rate it's roughly ~$80–$100 in raw frontier API-equivalent terms.
3 tips that helped the most for tuning a local Strata server:
1. Lock a strict harness first: enforce temp=0 bit-identical greedy parity checks + counter-balanced A/B runs (A->B->B->A) so thermal/cache drift never fools the agent into keeping fake wins.
2. Feed it ground-truth telemetry & disassembly, not just C++ source: turn on Strata's per-stage prefill/decode timers, sysfs power/clock stats, and objdump/profiler checks — that's how it caught hidden register spills and L3 cache thrashing.
3. Step in on memory-hierarchy trade-offs: the agent crushes inner-loop kernels on its own, while human nudges on big architectural trade-offs (like streaming cold KV from RAM to save VRAM for hot MoE experts) unlock the biggest jumps.
Surprisingly only the first ~3.5 hours went to setup, grabbing a community-maintained ROCm gfx906 toolchain (since AMD dropped official Vega 20 support in newer ROCm), and pulling ~64GB of model shards.
The remaining ~20 hours were pure iterative tuning and A/B benchmarking: isolating Xeon L3 cache ways, sweeping all 112 rocBLAS shapes, disassembling GPU binaries to eliminate register spills, writing custom Wave64/AVX2/FlashAttention kernels + inline GCN assembly, and running 256K context stress tests!
Haha 100% — $106 for >1.1 TB/s of 4096-bit HBM2 is basically a cheat code for MoE decode 😂
The real puzzle was surviving on just 16GB of it with zero Tensor Cores. Strata's hybrid CPU/GPU expert tiering + streaming QSA KV made it possible to pack 7,499 hot experts into that 16GB HBM2 while the $100 quad-channel Xeon chews through the cold tail. Seriously fun codebase to hack on!
One honest note + one wild detail I forgot to include above:
It wasn't 100% hands-off magic — I stepped in several times to interrupt dead ends, question bottlenecks (like why long-context decode initially dropped and whether cold KV could live in DDR3 instead), and steer the high-level direction.
Also, the craziest thing Gemini 4 Carbon did on its own: it unpacked and disassembled the compiled GPU ELF binaries (clang-offload-bundler + llvm-objdump / llvm-readobj) without being asked, and caught a hidden ROCm compiler trap — HIP translates CUDA's __launch_bounds__ per-SIMD instead of per-CU, silently spilling 552 registers to slow VRAM! Zeroing out all 552 spills + dumping and binary-patching the GPU's PowerPlay table from sysfs is what unlocked the biggest jump.
We're giving away a Hogwarts Legacy custom wrapped NVIDIA GeForce RTX 5090 to celebrate its official release with DLSS 5 🪄
To enter:
🟢 Share this post
🟢 Comment #RTXON
Terms and Conditions: https://t.co/45T3R732hf
I recently got an M5Stack Paper S3 — an e-ink dev board — on pure impulse. No plan, no idea what to do with it. I know almost nothing about embedded systems.
Two days later, with Claude Opus 4.5, I had a fully working e-reader: HTTP server, custom refresh rendering, a layout engine that handles EPUB with link navigation.
Three chat sessions. That's it.
My method was dead simple: make the AI write documentation explaining every design decision as it codes, dump everything into the next session, and if something's stuck beyond repair, just tell it to refactor from scratch.
But here's what actually got me thinking.
Modern industry delivers perfect black boxes. A Kindle is the ultimate example — engineered for the lowest common denominator of tens of millions of users. You pick it up, it works. Maybe you don't love everything about it, but it works. You're not supposed to understand how. You're not supposed to need to.
The Paper S3 is the opposite. It's a half-finished product — packaged screen, sensors, drivers — but no complete experience. It ships as a blank canvas.
And yet, with AI bridging the gap, I also didn't need to understand how the black box works. But unlike a Kindle user, I got to define my own experience. Every pixel, every interaction, every feature exists because I wanted it there — and nothing exists that I didn't ask for.
Here's the part that surprised me: building it myself was actually simpler.
Think about what Kindle has to deal with. DRM. Dozens of obscure file formats. A storefront ecosystem. Battery optimization pushed to absurd extremes. Accessibility compliance across regions. All of this engineered for hundreds of millions of users worldwide.
But do I actually need any of that? I don't use DRM. I don't buy from the Kindle Store. Half the formats it supports I've never even heard of. Yet I'm subsidizing the development cost of every single one of those features through the price of the device — features that were never mine to begin with, shaping an experience that was never designed for me.
With a dev board, I had none of that overhead. No compatibility matrix. No hypothetical user personas. No product managers debating edge cases I'll never encounter. I just needed it to read my books, in a form I find beautiful. That's it.
Commercial software has to keep doing addition to justify its valuation — piling on features, manufacturing needs that didn't exist before, expanding into adjacent markets. The product gets heavier, more complex, more "complete" — and paradoxically, less yours.
What I did was subtraction. Strip away everything until what's left is exactly what I need and nothing more. No more, no less. Just my existence, reflected in my tools.
I think this might be what AI coding actually means for people. It's not about turning everyone into a programmer. It's about returning the right to define the things around us — a right we quietly surrendered to the industrial machine in exchange for convenience.
Maybe one day, factories will only ship intentional blanks — half-finished canvases with the hard engineering done. AI provides the glue. And we provide the soul.
The product becomes fluid. It adapts to us, not the other way around.
To be continued.
Starting this Twitter as my personal notebook.
I’ve been exploring AI in generative creativity, and now I’m diving into a new journey: combining AI with Web3 to rethink how knowledge and copyright can be decentralized.
Just ideas, experiments, and reflections ahead.