Escha Labs

Minimum bits, maximum intelligence.

[Escha Labs @Eschalabs]

@AstraiaAI We just released a 2bit 35b, do give it a try!

[Escha Labs @Eschalabs]

@Mikerhinos Thanks. Unfortunately Escha is a custom format, we need to provide the decoder/runtime compatible with Llama.cpp.

[Escha Labs @Eschalabs]

Today we are introducing Escha-W2 quantization.

A 2-bit Qwen3.6-35B-A3B model built for fast, local inference. The complete model is 12.3GB on disk—small, enough to run on a single consumer GPU — while averaging ~100% of FP8 performance across 12 benchmarks, including:

MMLU-Pro: 80.9
MATH-500: 93.8
GPQA-Diamond: 77.8
LiveCodeBench v6: 62.6
BFCL tool use: 88.9
RULER 8K–128K: 89.9
Commonsense-6: 76.1

On a single RTX 4090, the model runs:
225 tok/s single-stream generation
on 12.3GB on-disk model size

Compressing a 35B MoE model this far without collapsing its capabilities required more than a standard quantization pass.

We built an end-to-end compression system combining state-of-the-art low-bit quantization with model-aware fine tuning and recovery to preserve capabilities most vulnerable to low-bit error.

Quantizing the Qwen 3.6 35B model - from the base model to the final deployable checkpoint — took approximately 10 hours to complete.

Escha-W2 runs through a custom Qwen3-MoE runtime, which includes the weight loader, low-bit decoding kernels and serving integration required to execute the Escha format efficiently. The runtime currently supports SGLang and ZML deployment. vLLM and llama.cpp will be supported in the next release.

No retraining from scratch. No specialized accelerator. One consumer GPU.

Model download: https://t.co/hbO3VuoDGU
Runtime download: https://t.co/hrTpO4FvuA
Apache-2.0 model and runtime.

[Escha Labs @Eschalabs]

You welcome! What we are confident is that this is version one release, hopefully it should be the worst version you'll ever see from us.

For short-term performance augmentation, we could quantize the RL/post trained variants of qwen, for example from thinking cap, Ornith, or BTL, all great options to bridge any performance gaps.

[Escha Labs @Eschalabs]

@brah_ddah for GLM 5.2, very rough back of the envelope is it will take between 200-400 H100 hours (SXM w 80GB) to quantize, and maybe another 1,000 hours to complete the evaluations. Not too bad actually...

[Escha Labs @Eschalabs]

@nariqwadet Escha 2bit is 12.3GB on-disk storage, that makes APEX Mini the closest comparable - can they publish the benchmarks so we can eye ball the two on a more apple to apple basis?

[Escha Labs @Eschalabs]

We publish our eval runs here.

The unweighted mean across 12 benchmarks was 100.7% of FP8, although results were not uniform: LiveCodeBench fell from 67.0 to 62.6. Other reported benchmarks are close to or slightly above the FP8 baseline, but some of the small gains should be interpreted as evaluation variance rather than actual quantization improvement.

Nothing like folks running their own evals to stress test it. Please share results with the community. We want to hear the good and the bad. There is nothing to hide.

[Escha Labs @Eschalabs]

@vitalune7 Someone did, but love to see more. We encourage the community to stress test this on real tasks and share the findings, good and bad, so we can improve:

https://t.co/ey7uikFkj0

[Bart Mol @Bart_Mol]

I independently tested Escha-W2 on my RTX 3090 against the same Qwen3.6-35B-A3B model in regular Q4 GGUF.

It fits on my RTX 3090 with room to spare; it's faster and delivers comparable quality to the Q4 model.

MATH-500 subset (100 problems, thinking enabled, 16k-token budget, temperature 0):
- Escha-W2: 77/100 in 35 minutes
- Q4 GGUF: 78/100 in 3 hours 39 minutes

Almost every failure in both models was due to hitting the token budget: when a model completed an answer, it was correct.

The 2-bit compression did not materially hurt reasoning.

HumanEval+:
- Escha-W2: 87.8
- Q4 GGUF: 89.6

Small difference in quality, big difference in size and speed:

- Q4: 23GB, fills the 24GB card, 88 tok/s
- Escha-W2: 12.3GB, 155 tok/s single-stream
- Escha-W2: 400+ tok/s across eight parallel streams

That is roughly 1.8× faster for a single stream and more than 4.5× the total throughput when batching. The Q4 leaves practically no VRAM available for batching; Escha does.

This means I can run a 35B MoE model locally on my 3090 with room to spare for context or other models. Really cool.

[Escha Labs @Eschalabs]

@APMI93301887 @Bart_Mol We provide a custom runtime for the model. You can download it here: https://t.co/1rNwwYPwGY

[Escha Labs @Eschalabs]

@soup233328077 Wait for the Qwen 27B release, you will be able to run it with 12GB vram.

[Escha Labs @Eschalabs]

@wait_sasha Not yet, MLX first, then we will try to figure out the best way to bring AMD online

[Escha Labs @Eschalabs]

We evaluated the model across 12 capabilities across common sense, math, coding, instruction following, etc, the average across the 12 benches is 100% of the FP8 baseline. LiveCodeBench didn’t do quite well, but others did a little better. We released this so the community can give it the proper test and eval more comprehensively than we could ever do and draw their own conclusions.

[Escha Labs @Eschalabs]

@jetha @miguelsaavedra @Hikari_07_jp Please try it and let us know the good and the bad!

[Escha Labs @Eschalabs]

@jtcera Appreciate it! Give us a bit of time, we can make it faster :)

[Escha Labs @Eschalabs]

@FredFRodrigues Yes, getting MLX ready, hopefully by next week

[Escha Labs @Eschalabs]

@jtcera Please post when done

[Escha Labs @Eschalabs]

@alew3 Yes sir, any GPU with 24+GB of vram should be able to handle 256k context. TTFT might be slow, please give it a test.

[Escha Labs @Eschalabs]

@emilstridell Nice catch, will fix it up :)

[Escha Labs @Eschalabs]

@JosephJacks_ Fractional sub 1bit is a thing, so even cheaper hardware!

[Escha Labs @Eschalabs]

@nariqwadet Looks great! But where do we lose?

[Escha Labs @Eschalabs]

@agentic_austin Sorry, we are working on MLX right now so you can run on Mac