AboveSpec

Love 3d printing, playing with local llms and learning Claude Code

[AboveSpec @above_spec]

Qwen3.6-27B-MTP at ~61 tok/s. 100k context.
On two *used* RTX 3080 Tis — not the RTX 3090 everyone benchmarks (24GB, but split across 2 cards on PCIe 3.0 x8/x8, no NVLink).

Running llama.cpp's new MTP speculative decoding. The deep-context bottleneck? Nobody's talking about it. 🧵

[AboveSpec @above_spec]

@afcmax22 Less EMFs for wired headphones = healthier for your brain. Come on, everyone should know that?

[AboveSpec @above_spec]

@witcheer Won't hot air from the AIO liquid cpu cooler be sucked right into rtx 5090? Maybe it'll be fine if you don't run sustained 575/600 watt loads.

[AboveSpec @above_spec]

@TheRussiaBlog @cuggxa Should be something like Добрую мысль ты сказал****

Hello from Toronto!

[AboveSpec @above_spec]

@JoesInvestments If it's regular 2x3080 you have 20GB of VRAM, which is not quite enough for this quant+MTP. You would need to try one of the q3 versions probably to fit all in. If you have 3080 12GB, then this should work exactly the same.

[AboveSpec @above_spec]

Qwen3.6-27B-MTP at ~61 tok/s. 100k context.
On two *used* RTX 3080 Tis — not the RTX 3090 everyone benchmarks (24GB, but split across 2 cards on PCIe 3.0 x8/x8, no NVLink).

Running llama.cpp's new MTP speculative decoding. The deep-context bottleneck? Nobody's talking about it. 🧵

[AboveSpec @above_spec]

@idare split layers

[AboveSpec @above_spec]

@1337hero Cable management has never been my strong suit, lol

[AboveSpec @above_spec]

If you have another agent like Claude Code running from cloud you could ask it to inspect logs of what's happening. It could probably find the cause, do you track temperature of your 3090s, they might be overheating. For me usually this only happens when I run out of memory (OOM).

[AboveSpec @above_spec]

@EvilOni Nice, I did see them on ebay recently. How much did you get them for?

[AboveSpec @above_spec]

It looks like your settings are good. I guess q5_k_xl is slower in generation than q4_k_m. You could try --spec-draft-n-max 3 instead of 2, but don't think it will make that much difference. I haven't tested 2 x 5060 ti on 27b myself. I did test a more compressed Qwen3.6-27B IQ3_K_R4 with no mtp on a single 5060 ti and got like 28 t/s which I managed to increase to 33-34 with aggressive memory overclocking: https://t.co/0wGfoTk8LE

[AboveSpec @above_spec]

@MakJoris @therealazzurro @therealazzurro is getting that with 2 x 5060 ti. But we don't know his exact setup, perhaps there is room for improvement. Your 37t/s is extremely good for 5060 ti! I got 60 for 2 x 3080 ti but it does go down to like 40 at longer context.

[AboveSpec @above_spec]

@MakJoris @therealazzurro Are you using ik_llama.cpp or mainline llama.cpp?

[AboveSpec @above_spec]

@ItsmeAjayKV @threejs They are cool, they liked my post when i posted a coding example using three.js! 👍

[AboveSpec @above_spec]

@Hihiohoo No just regular 12gb versions

[AboveSpec @above_spec]

@Gianluca_Bing Of course

[AboveSpec @above_spec]

@therealazzurro You can probably squeeze out more tokens out of 2x5060 ti 16G. Q5_k_xl gives you better quality for sure. Check a comment above yours -- 2 x 9060XT 16Gb doing 74tps.

[AboveSpec @above_spec]

@i_okk Amazing! Dual 9060XT must be the most affordable way to get 32gb of Vram!

[AboveSpec @above_spec]

@NVenetias Are you running same quant for the model? I am also using q4 compression for kv cache here. With 48gb of vram you should have more than enough to run full 262k context.

[AboveSpec @above_spec]

@callegariai Yes, you can do the same on a single 3090 with less power consumption

[AboveSpec @above_spec]

@ProofOfPrints Haven't tested it. Ofc claude opus will be superior, but yeah interesting to try claude for planning and local qwen for execution!