I like too many things and I know I know too little, but I boldly call myself a polymath.
I run new AI on old hardware, or whatever I have available. πͺ‘
I test cheap hardware, AI models and quants and I report the results on the link in the bio. No expensive fancy DGX, Halo, RTX 6000 hardware, just real people and accessible hardware is used. Real results on real agentic tasks are what measures a model usability for us peasants
I'm working on my first quantization, I want to make MoEs accessible for real people and tech enthusiasts, not only people with access to expensive hardware.
First model will be the Laguna S, which is going down to 33GB full model size. Testings ongoing on the asymmetric quant