We released experimental MTP Qwen3.6 Unsloth GGUFs!
Qwen3.6 27B MTP now runs at 140 tokens/s. Qwen3.6 35B-A3B MTP gets 220 tokens/s generation on a single GPU.
Qwen3.6 27B and 35B-A3B have >1.4x speed-up over the original GGUFs without any change in accuracy.
Guide + GGUFs + Benchmarks: https://t.co/x9BYC3iXCL
In terms of average speedup, we see a 1.4x for dense models at draft tokens = 2 and for the MoE around 1.15 to 1.2x.
We do not recommend more than 2 draft tokens because the acceptance rate drops precipitously from 83% to 50% with 4 draft tokens, and the forward passes for MTP become less beneficial.
Use `--spec-type mtp --spec-draft-n-max 2`
Thanks to Aman for https://t.co/0WKkIC0kyW!
As I conjectured, Qwen3.5 follows exactly a layer-pruning recipe: prune the last 25% of layers (the most cursed layers), and then recover with KD. This directly supports my view that the next scaling gains may come not only from adding depth, but from using depth better. @shengkun_t52337 Thanks for making such a detailed report!
Great read from the @RedHat_AI team — a comprehensive investigation into TurboQuant in vLLM, with FP8 and BF16 as reference baselines: 4 models (30B to 200B+, decoder-only and MoE) and 5 benchmarks covering long-context retrieval and reasoning, all on the stable vLLM 0.20.2 release.
If you're considering TurboQuant for your workload, this is the data to start from.
📝 https://t.co/YkQIsyuDUN
OpenCode x DeepSeek V4 Flash - free for a limited time
DeepSeek V4 Flash is currently our most popular model in Go
Give it a try if you haven’t already
🔥DeepSeek Input Cache Price Drop!
Effective immediately, the price for input cache hits across the ENTIRE DeepSeek API series is reduced to just 1/10th of the original price! Build more efficiently for less.
📌Reminder: The DeepSeek-V4-Pro 75% OFF promotion is still active until May 5th, 2026, 15:59 (UTC Time).
🚀Meet Carnice-V2-27b🚀
→ Carnice is a 27 billion parameter model capable of beating models 10x the size in Hermes-agent, fully open-source and built on top of Qwen3.6-27B
→Build to fit on Consumer GPU on a 3090+ to run locally🔥
→Carnice-V2-27b is the successor of Carnice-27b trained on more and better data
Download it here: https://t.co/zHKzbZxLkM
Download GGUF here: https://t.co/bGRjrlXMSg
thanks to @NousResearch@LambdaAPI for making this possible.
🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length.
🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models.
🔹 DeepSeek-V4-Flash: 284B total / 13B active params. Your fast, efficient, and economical choice.
Try it now at https://t.co/GCdiMzk1Dl via Expert Mode / Instant Mode. API is updated & available today!
📄 Tech Report: https://t.co/drlDrxkYtp
🤗 Open Weights: https://t.co/T13Y8i7SDM
1/n
BOOM!
New open source model!
We are testing this now at The Zero-Human Company!
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled.
This Frankenstein of an AI model runs fast local consumer hardware!
More soon! https://t.co/qILhHgWVao