JustVugg

Founder of Colibrì a tiny engine for immense model for affordable hardware. 🐦

[JustVugg @justvugg]

working on glm5.3 flash on colibrì…

[JustVugg @justvugg]

@jozzzeey Colibrì and nothing about me is involved in fee or others things. Colibrì is a opensource project.Nothing else!

[JustVugg @justvugg]

@ArifiLabs @TeksEdge We are working every day to improve the most important points. Naturally, the current models have a structure that Colibrì has changed. We will always improve more.

[David Hendrickson @TeksEdge]

🔝 New and ⬆️ improved Colibrì 👉 just run the model - don’t worry about the memory.

👀 Runs Qwen3.6, DeepSeek V4 Flash, GLM-5.2 and Kimi K3!

Real LLM Virtual memory!

Back in July, Colibrì was the crazy pure-C engine that could run GLM-5.2 744B by streaming MoE experts from SSD.

👾 Now it has evolved into something much more interesting.

🎯 Colibrì treats your whole machine like one giant AI memory hierarchy...
🎮 VRAM → hottest experts
🧠 System RAM → warm experts
💾 NVMe SSD → everything else

And now it supports models from ...
🔥 Qwen3.6-35B-A3B
🔥 DeepSeek V4 Flash 284B
🔥 GLM-5.2 744B
🔥 Inkling 975B
🔥 Kimi K3 2.8 TRILLION parameters

Plus:
⚡ GPU expert caching
🔮 expert prefetching
💽 dual-NVMe striping
🟢 CUDA
🔴 ROCm
🍎 Metal
🟣 Vulkan
🌐 distributed expert workers

One benchmark showed expert caching taking Qwen3.6-35B-A3B from 1.44 → 10.05 tok/s on two 8GB GPUs.

And the documented RAM floor for GLM-5.2 is now just 16GB.

This feels less like model offloading now and more like a virtual-memory system for gigantic Local AI models. 🔥

🔗 GitHub /JustVugg/colibri

[JustVugg @justvugg]

@TotallyArea @TeksEdge Try this because the first have this problem!
This is the official now 👍🏻👍🏻
https://t.co/pLV3FezyTt

[JustVugg @justvugg]

If someone dedicates time to you, your project or your ideas you can only be grateful.

[JustVugg @justvugg]

@NilsReimelt Try Kimi 3 too there a lot of improvements

[JustVugg @justvugg]

colibri v1.8.0 is out. Frontier MoEs now waste less work across SSD, RAM,CPU and GPU—and the GPU you bought in 2016 is back in the game. DeepSeek V4's CUDA tier now runs on GTX 10 and RTX 20 cards: a 284B model with accelerated prefill. A short thread.

[Nils Reimelt @NilsReimelt]

@justvugg colibri 1.8 works great! And with DeepSeek results are much better and faster than GLM and reasoning (solving complex logic puzzles) is now correct and reliable.

[JustVugg @justvugg]

@NilsReimelt Thanks @NilsReimelt 👍🏻👍🏻🙌🏻🙌🏻

[JustVugg @justvugg]

60 PRs. 13 community contributors. Every risky feature is opt-in; default paths are protected by token-exact oracles. Frontier-scale models on hardware you own:
https://t.co/jkURdQvTf0

[JustVugg @justvugg]

The KV cache learned FP8 and KV_TQ. coli tune now measures every engine, tests rotating prompts, and derives safe RAM and expert-cache caps from the actual machine. The v1.7.0 regression that made Linux reject valid DeepSeek V4 CUDA builds is fixed too.

[JustVugg @justvugg]

@0x0SojalSec This is the standard we use in colibrì try this

https://t.co/pLV3FezyTt

[JustVugg @justvugg]

@tom_doerr Thanks @tom_doerr 👍🏻👍🏻

[JustVugg @justvugg]

Thanks to kreuzzelg, monotophic, terrizoaguimor, ZacharyZcR, dcutugno, RDouglasSharp, bherald, SebaWag, outtodata, lineape, aaristov, benmaster82, zh-Processor, 8PotatoChip8, weber-software, and everyone who filed a datapoint.

Tiny engine, immense model. 🐦

[JustVugg @justvugg]

colibrì v1.7.0 is out.

A sixth model family, its GPU tier, and the expert matmul path rebuilt from scratch. Every step bit-identical to what it replaced.

80+ PRs since v1.6.2, from 15 contributors.

https://t.co/qs7jTvVk7H

[JustVugg @justvugg]

Elsewhere:

DeepSeek V4 loader lanes 3 to 9, for 1.41x decode on the real 156 GB checkpoint
DeepGEMM sm120 fetched at a pinned commit, 2.5x prefill, nothing vendored
Every engine now shares one serve framing codec