Qwen3.8-27B on a single RTX 3060. after weeks of testing quants and configs, this is where I'm stopping -- these two are my final builds for this card. 158K ctx at 22 tok/s, or 82K at 38 tok/s. (config & weights below)
tested four IQ2 quants across seven benchmarks and GSQ-RCO IQ2_XS won, while being the second smallest file of the four.
the catch: GSQ-RCO ships no MTP head. you borrow Unsloth's -- same base model, drops straight in -- and pay ~76K of context for 1.7x decode. prefill is ~460 tok/s either way, MTP only touches generation.
which one is "worth it" depends entirely on what you run. short few-shot tasks never touch 64K, so the extra context is dead weight and you just want the speed. long single-session agent runs blow past 82K and no decode rate saves you once you're out of context. that's why both configs exist.
everything on low reasoning effort. at 2-bit the higher settings spiral instead of terminating.
🐢 context (158K):
llama-server -m Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf -ngl 99 -c 158000 -fa on --jinja -np 1 --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on --chat-template-kwargs '{"reasoning_effort":"low"}'
⚡ MTP (82K):
llama-server -m Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf -md mtp-Qwen3.8-27B-Q4_0.gguf -ngl 99 -c 82000 -fa on --jinja -np 1 --spec-type draft-mtp --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on --chat-template-kwargs '{"reasoning_effort":"low"}'
Nvidia Dropped Qwen 3.8 Flash and it’s over 60% smaller now with minimal loss making this probably the BEST model for single Spark users. Do you yall want me to cook up a recipe tonight ? https://t.co/0jUCAayKEo