Lossless weight compression has saved LLMs GPU memory, but not time. Decoding costs more than it saves.
NyoomFloat12 (NF12) is a lossless 12-bit BF16 format with decode cheap enough to speed up inference. Qwen3-32B: 1.23× faster token generation, up to 2.54× rollout throughput🧵