@murasametech before you know it, you've updated the ram in your 1st system (moar larger models please) and you think, what should I do with this old ram? And bam, now you have 2 ai inference systems, haha
Loving this new inference engine (strata based) for running larger models that normally wouldnt fit on my hardware. Running GLM 5.3 Flash on my AMD EPYC server (7702 cpu). Look at all those cpu threads along with GPU usage. https://t.co/3FkdZuj2PR #LocalAI#AI