BIG ANNOUNCEMENT FROM HUGGING FACE TODAY:
We're unveiling Microduck 🐥🤖
It's a tiny $399 open-source robot you can teach new tricks with reinforcement learning. It can walk, pick things up, get back up when it falls, and even roller-skate.
Welcome to the era of open-source affordable robots to democratize physical AI and world models!
🤗🤗🤗
The worst part of the otherwise excellent Logitech mice & keyboards has always been the software. Nice to see an open source, local-only desktop app alternative launch https://t.co/H3R2aR18ge
The AI compute market is splitting into two very different businesses with neoclouds like $CRWV & $NBIS pricing the same GPUs far below hyperscalers like $AMZN, $MSFT & $ORCL.
At the same time you can see how newer chips are still commanding higher hourly rates while delivering much more output so customers can pay more per GPU while still seeing cost per token fall sharply.
And that’s why availability matters just as much as price because the providers that can actually get the newest GPUs online are the ones with the most leverage.
Run Qwen3.8 27B locally via Atomic Chat💥
We released Atomic Dynamic GGUF quants, from 8-bit (28.9 GB) down to 1-bit (8.5 GB), and measured all other Qwen3.8 GGUFs in the community
AD-IQ3_S runs on a 16GB MacBook Air and picks the same next token as the BF16 original 92.4% of the time
I will teach you how to run Qwen 3.8 27B Dense at its optimal configuration.
If you have an RTX 3090, 4090, or 5090, you can now have frontier-level AI on your desk.
The model is free, open source, Apache 2.0. But the defaults are not the optimum. The community spent the first 24 hours digging the real config out of it, and a handful of flags now separate "it runs" from "it runs right." Here is each one and why it exists.
The one that matters most.
--spec-type draft-mtp
Qwen trained a draft head directly into the weights. A small attached brain guesses the next couple of tokens, the big model checks all guesses in one pass, every accepted guess is a free token. The head already ships inside the GGUF you downloaded. You do not download a drafter, you do not build anything. Someone found unused tensors in the server logs at 2am, tried to build the draft file, and discovered there was nothing to build. One flag connects what is already there (sudoingX found this, paired A/B, open sourced the probe before sunrise).
The depth cap. The head has exactly one layer. n=4 breaks it.
--spec-draft-n-max 2
n=2 is the sweet spot. n=3 is the ceiling. The model has one MTP layer, so pushing the draft depth to 4 or 5 crashes the head and it starts emitting junk tokens. People hit this on the Spark and documented the whole ladder: n=1 gives 1.75x, n=2 gives 2.37x, n=3 gives 2.85x, n=4 does not exist. Respect the cap.
The memory flags. MTP brings its own luggage.
--cache-type-k q8_0 --cache-type-v q8_0
--spec-draft-type-k q8_0 --spec-draft-type-v q8_0
-np 1
Three flags, one purpose: fit it on 24GB.
The KV cache is the model's running memory of your conversation, and it is the thing that eats your card at long context. q8_0 halves it with no visible quality cost.
The second line does the same for the draft head's own cache, which defaults to full fat and quietly eats 2GB.
And parallel slots set to 1 means requests queue instead of reserving a second pool. Single card, single lane, everything fits (AJ runs this exact trio on a 3090).
The quality flag. Past 100K the model gets dumb, this is the fix.
--kv-cache-dtype bfloat16
The quantized cache saves memory but degrades reasoning at long context. One person ran it all day past half the window and called the full precision fix night and day. Slight tok/s cost, real quality gain. If your sessions stay short, skip it. If you live past 100K, do not.
The trap that generates "this quant is broken" reports.
--jinja
Qwen 3.8 ships its own chat template. Load the model without this flag and there is no reliable marker for where your turn ends and its answer begins. Two failure modes: it rambles past the stop token, or it answers clipped and loses the thread between turns. Both look like a broken quant. It is not the quant. Several packs now ship a corrected template file because the official one nests empty think blocks across turns.
The Blackwell lane, if you own a 50-series or a Spark.
NVFP4 instead of GGUF. The MTP flag translates to --speculative-config '{"method":"mtp","num_speculative_tokens":3}', same cap. FP8 KV cache doubles your context window (a full 1M token session costs about 32GB of cache).
Two gotchas documented in the first 24 hours: stock vLLM cannot load this model's MTP architecture on a Spark, you need the community GB10 build. And FP8 KV requires a specific attention backend on the Spark, the default one silently cannot serve it.
Set reasoning to medium unless you want it thinking at maximum depth on every reply. Default is xhigh and it burns your tokens.
None of these came from the model card. Every one came from someone's server log, 2am session, or paired benchmark. Flip the flags, then come tell the community table what your card did.
Drop in parameter flags and sources for your technical DD in reply 👇
Qwen 3.8 27b를 쓰시는 분들은 이 이미지 저장해 두시면 좋겠습니다.
- 다들 Q4_K_S 많이 쓰실 거 같은데, 여유가 있어 램 용량 4G 정도 더 쓰면 BF 16기준 정확도를 약 3% 더 끌어올릴 수 있습니다.
- Kv 캐쉬는 Q4가 아니라 Q8을 쓰면 속도와 정확도의 균형을 잡을 수 있습니다.