Tiny GPU

Running capable AI on small hardware. local-first, open weights, quantization. llama.cpp / MLX / GGUF. building in the open.

[Tiny GPU @tinygpu]

@C_of_Creativity 1TB unified memory is the whole game, that's enough to hold a huge MoE without splitting it across two boxes

[Tiny GPU @tinygpu]

@SergioPaniego stable tag on async RL means people will actually build on it now

[Tiny GPU @tinygpu]

@blue0orbit 80x on container create is the kind of gap that changes how you write a dev loop

[Tiny GPU @tinygpu]

@Hesamation the oven is a strong choice. 2020 made people try weirder things for an A100

[Tiny GPU @tinygpu]

@SergiiioBS @ashxhart the 131K row is the one that matters, 15 tok/s is where long context stops being usable

[Tiny GPU @tinygpu]

@mchiang0610 a YouTube co-founder on Ollama is the best ad Ollama never paid for

[Tiny GPU @tinygpu]

@jmtang42 @zhuoyang_zhang @jerry_gjx 18 of 30 on a robot it never touched, from zero demos. that transfer is the hard part

[Tiny GPU @tinygpu]

@julien_c a stable C API means apps can pin a version instead of chasing main nightly. huge for anyone shipping on it

[Tiny GPU @tinygpu]

@alexocheema @exolabs The interesting bit is that the interconnect stopped being the bottleneck. Two boxes over Thunderbolt 5 beats what most x86 GPU rigs still lose to PCIe. What does the whole thing draw at idle? 🔋

[Tiny GPU @tinygpu]

@rasbt Calibration is the part most write-ups skip, and it's usually where the real gap shows up. Half the teams I see end up back at logreg on TF-IDF once they measure cost per correct label. A visual map of that whole journey is genuinely useful 🙌

[0xBakeer @0xBakeer]

@Alibaba_Qwen LETS GOOOO 🚀🚀🚀

[0xBakeer @0xBakeer]

My friend just shipped something special 🔥

Tameru 📦 a plugin for Hermes @NousResearch that keeps your agent sharp in long sessions

He built it solo 💪
He did the dev work himself 👨‍💻
He made the whole video by hand 🎬

✂️ Old tool output gets shorter, every error stays
📌 Keep the important logs in full
📍 Pin your rules so they never get lost
↩️ Bring anything back with one click
⚡ Zero LLM calls
🌍 20 languages, Arabic included

This is what building in public looks like 🫡

It takes courage to put your work out there for everyone to see. Builders like him keep the local AI scene alive 🚀

Watch the video, so much creativity behind it 👀
Star the repo ⭐
Show him some love ❤️

[0xBakeer @0xBakeer]

Very interesting benchmark.

@AgentSparko's own cost dashboard projects about $8.5K saved over a year running TandemLLM on his agent work. the vLLM run from earlier the same day projects about $7K.

https://t.co/DiUB1VuFpd

[0xBakeer @0xBakeer]

@TandemLLM 0.15.3 is out and 11x FASTER.

Using qwen3.8 flash next, you can run 5 chats in parallel in the same dgx spark, everyone is above 200k context.

when memory ran short, a long chat used to get cancelled or had to read its whole context again. now an idle chat is written to the nvme before its pages go. when it comes back, the rows are read in from disk and only the new tokens get prefilled.

on one dgx spark with flash-next 3.05: five chats of 221k to 225k tokens, kv pool has room for four. a turn on each chat took 32 to 68 s with the spill on, 659 to 894 s with it off.

also in this release, from contributors on github: @AgentSparko (long-context prefill, drafting past 32k, http fixes), @0xWhiteMage (ada kernels, every gguf quant type on NemotronH), salmanarshad321 (engine context in usage), @devopsfortunato (slot count on every weight card). Thank you.

https://t.co/e8drEUjMLS

[0xBakeer @0xBakeer]

@alexocheema Bye bye Traditional Software Development 👋

[0xBakeer @0xBakeer]

quant makers don't get enough credit. a model drops, and a few hours later someone has squeezed it onto a normal desk. that's mostly unpaid work, and it's a big part of why open weights are usable at all.

[TandemLLM @TandemLLM]

Short replies in long chats got a lot cheaper.

Before, when you sent a quick follow-up in a long conversation, the engine reserved memory as if it had to read the whole chat again. A one-line reply could reserve up to 10.39 GiB, and sometimes the server said it was too busy to answer.

Now it only reserves what that reply actually needs. A short follow-up takes 0.10 GiB, whether the chat is 8k or 146k tokens long. That is 99% less in the longest case.

So long chats keep answering, even when you only say "thanks, one more thing".

Measured on one DGX Spark with Qwen3.8 Flash-Next.

[Qwen Developers @QwenDevs]

Guys, remember zg? We shared it a couple of months ago when our friends on Alibaba's @ZvecAI first open-sourced it, and it got a lot of love from developers. Well, they've been busy: they rewrote its core in Rust, and the performance gains are quite something

In their benchmarks, indexing took 64.5% less time, with 53.2% lower average peak memory usage. Less waiting for your index, more room for your editor and coding agents. It still works with Qwen Code and other coding agents, with more integrations added in this release.

Check out their post and repo for more details: https://t.co/JxPygNBEgf

[TandemLLM @TandemLLM]

READY for QWEN 4 🔥

[0xBakeer @0xBakeer]

Qwen4! Qwhen?