w4a8 format for MiniMax-H3
takes my 8gb vram card about 35 minutes to gen with the regular INT8.
took my card about 7 minutes to gen with this model. not even kidding.
the quality is there, the adherence is there. try it for yourself!
https://t.co/xBtIuXdGvq
Kijai's Minimax H3
compressed model weight
He achieved this by squeezing the model's weights down to 4-bit (saving massive amounts of space) but running the math in 8-bit (keeping it fast and accurate) using a special layout called AsymW4A8Int8Layout
👇
https://t.co/I4oh4aW1A2