@0xSero I simply want it. I want to simply tired to pay cheap inference and reach my cut off and have to stop working. I just want to be able to run decent local llms like qwen3.8-27B and not have to worry about how many tokens I have left.
Qwen3.8-27B EXL3 is the best model you can run on your 16-24 or 32 GB VRAM GPUs.
Gives you great quality + large context.
That includes RTX 5060 Ti, 5070 Ti, 3090, 5090 and many others Nvidia GPUs.
Try it with a simple one-click install:
https://t.co/740Y19fmsu
@ArmanHezarkhani I see people complaining that you use @karpathy face for this, I am actually glad you did. I probably wouldn't have look at this if it wasn't for it, and I am glad I did because it is a great resource of info. Thanks