Michael Helcig

Working on local intelligence.
CS @ ETH Zurich, Research @ ISTA | EPFL
Co-Creator of GSQ-RCO

[Kyle Hessling @KyleHessling1]

ISTA DAS labs is on X and has spoken!

These are the geniuses behind the incredible Qwen 3.8 Flash Next IQ_3S quants I have been posting about!

Please follow this account @mhelcig, it will be highly valuable moving forward, and @coldniko if you haven’t already!

Godspeed to the team!

[Michael Helcig @mhelcig]

@coldniko @adotdad Cheers to that, Niko. Let's go 🙌

[Michael Helcig @mhelcig]

Our largest model yet.

763B parameters, compressed with GSQ-RCO and co-optimized with Strata (by @coldniko) for local inference.

On Hugging Face this Friday.

Much more is coming. Follow us for future releases.

[Alireza Dadgarnia @adotdad]

We've updated Qwen3.8-Flash-Next-GSQ-RCO-Coder! Improved RCO and more diverse calibration data boost its agentic coding and vision capabilities. We also pushed the model to its full 262K context window through multiple compactions without encountering any looping behavior.

[Michael Helcig @mhelcig]

@ItsmeAjayKV We recently also published Qwen3.8 Flash-Next GSQ-RCO Coder specifically for coding workloads:

https://t.co/p0KKCFwlfg

[Michael Helcig @mhelcig]

Qwen3.8-Flash-Next (176.9B MoE, 354 GB BF16) quantized with GSQ + RCO.

3.5 bpw, 83.6 GB: matches BF16 (avg 93.26 vs 93.12).

Coder build drops half the experts, 1.89 bpw effective. 29.6 GB resident, fits a 32 GB GPU. LiveCodeBench 86.3 vs 87.4 (BF16).

https://t.co/9eBua0v1Iy

[Michael Helcig @mhelcig]

5/ Coder is experimental. Requests for other models welcome.
Coder: https://t.co/gukCHrArQz
GSQ: https://t.co/frEP5XCMNi
RCO: https://t.co/HZ74WQwUCk

[Michael Helcig @mhelcig]

Qwen3.8-Flash-Next (176.9B MoE, 354 GB BF16) quantized with GSQ + RCO.

3.5 bpw, 83.6 GB: matches BF16 (avg 93.26 vs 93.12).

Coder build drops half the experts, 1.89 bpw effective. 29.6 GB resident, fits a 32 GB GPU. LiveCodeBench 86.3 vs 87.4 (BF16).

https://t.co/9eBua0v1Iy

[Michael Helcig @mhelcig]

4/ Of the 58.4 GB, only 29.6 GB must stay resident. The rest is an n-gram lookup shard that can be served from disk. Coding retention: LiveCodeBench v6 98.7%, SWE-bench Verified 91.3%, both at xhigh reasoning.

[Verda @verdacloud]

The 99.8% figure is the average across AIME25, GPQA-Diamond and LiveCodeBench v6. IQ3_S matches BF16 on AIME25 and LiveCodeBench v6, and trails by 0.51 points on GPQA-Diamond.

The release drew nearly 885K downloads last month. Credit @mhelcig , @DAlistarh and team.

[Verda @verdacloud]

We’re proud to have supported the researchers at ISTA-DASLab with compute resources for their GSQ-RCO quantizations of @Alibaba_Qwen 3.8-27B.

The release reduces the model from 53.8 GB to 11.8 GB while retaining 99.8% of the BF16 model’s reported benchmark performance.

[Doğukan @DogukanUrker]

Qwen3.8-27B at 2-bit, and GSQ-RCO is crazy good. PatchEval, my agentic coding bench -- 20 tasks, Pi agent, 1 repair attempt.

GSQ-RCO IQ2_XS 16/20, AtomicChat IQ2_XXS 7/20. Same on my three.js bench, where it also beat Unsloth.

The hosted Qwen API run got 16/20 too -- a 2-bit quant on a 12GB 3060 matching whatever they're serving.

[Benjamin Marie @bnjmn_marie]

An excellent Qwen3.8 27B GGUF under 10 GB:
https://t.co/otx7Br7NOk

Probably the best we can have at that size using state-of-the-art quantization techniques.

Both IQ2_S and IQ3_XXS are OK.

I would avoid using IQ2_XXS, especially if you want to use it for coding. Its accuracy on LiveCodeBench is slightly below Qwen3-4B-Thinking-2507.

[Eldar Kurtić @_EldarKurtic]

Our new paper on statistically lossless quantization is now available on arxiv and will appear at COLM.

[Red Hat AI @RedHat_AI]

Lossless quantization has usually meant giving up inference speedup. This paper changes that.

SLQ (Statistically-Lossless Quantization) reaches task-lossless compression at 3.3 bits per parameter, and distribution-lossless at 5-6 bpp where the output distribution is practically indistinguishable from the original.

1.7 to 3.6x throughput over BF16 in @vllm_project. Beats FP8 while staying lossless.

From Michael Helcig, @_EldarKurtic, and @DAlistarh.

https://t.co/DBsAvlMVMl

[Dan Alistarh @DAlistarh]

Weight-only quantization powers local LLMs like llama.cpp or Ollama. But SOTA quantized accuracy requires complex kernels that are notoriously hard to implement.
Can we get SOTA accuracy and keep things simple? Our new GSQ (Gumbel-Softmax Quantization) method says yes. 🧵

[Dan Alistarh @DAlistarh]

We're releasing the DASLab GGUF Quantization Toolkit! 🚀
First open-source toolkit bringing GPTQ + EvoPress to @ggerganov's GGUF format, enabling heterogeneous quantization based on importance.
Result: Better models at the same file size.
[1/5]