@PrismML@ollama please add support for PrismML’s Bonsai 2 PQ2_0 / PTQ1_0 GGUF formats.
Right now Ollama fails on "ollama show" with:
unsupported tensor "output.weight" size overflows
and Hermes/tool-calling also gets rejected as “does not support tools.”
I'm using ollama 0.34.2 rn.
@witcheer I like this idea better cause for low end users like me, running on 64k context and the local models eating up 25-35k context right away before starting to write anything.
@CaryPalmerr@Teknium@elonmusk my LinkedIn account got hijacked a few days ago and turned into a random woman's profile till i noticed it and recovered it.