Other configs:
•split across 2 cards: 25-29% slower (default split cuts the looped layers)
•whole loop on one card: better, still slower than one card
•DSpark drafting: no help
•Strata's new native engine: 50 tok/s, 32k max on 16GB. Its 2-card mode ran 1.3 tok/s 💀
Inkling 2 Mini update after a couple days on my Lenovo P520 (5060 Ti 16GB + 3060 12GB).
Short version: slower than Qwen Flash Next and burns more tokens on hard tasks. Unless you're really RAM starved, not worth it IMO.
🧵 for more.
Speed: best setup was a 2-bit quant on the 5060 Ti alone w/ the newest llama.cpp. 60 tok/s short, ~35 at 114k, full 262k context on one card. Flash Next on the same card: ~61 at 113k.
I’m getting unimpressive speeds on Inkling 2 mini with longer context/larger window. Qwen 3.8 Flash Next is more than 2x faster.
System: Lenovo P520, Xeon W-2255, 232GB DDR4, RTX 5060 Ti 16GB + RTX 3060 12GB
@RedFasterX@MythomancerX On very short prompt/low context using unsloth’s IQ3_XXS
the split across both cards costs significantly - about 25% of generation speed (55.6 down to 41.6 tok/s) and about 9% of prompt reading.
@MythomancerX Ran it apples to apples: same prompt, ~114k tokens. Flash Next 60.8 tok/s decode on the 5060 Ti alone vs Inkling 25 on 5060 Ti + 3060. Prefill 1,337 vs 249. Flash Next barely slows with context (~10%), Inkling drops ~37%.
@scaling01 9b Qwen on a bunch of refrigerators is less concerning than Mythos-class models combining subtle data patterns (eg steganographic signals in code, blockchain data) for coordination, long-horizon memory, injection, and even weight fragment distribution.
The Unsloth Qwen 4B, 9B and both 27B quants, ISTA 27B, and Gemma models used MTP (multi-token prediction); both Flash Next quants used Strata with MTP and GPU expert caching. Jackrong, Ornith and DavidAU Twin Turbo used no speculative decoding, while Spark used its own llama.cpp
@elchiapp Yes, when I looked at the numbers, they were not impressive compared to Qwen. But if you believe in open source another competent lab should be welcomed with open arms.