@gooncultist They have you install some Python stuff, but it's only to covert models from one form to another. Once the 4-bit quantized formats are distributed widely, it'll just be (1) download program, (2) move model to folder, (3) write prompt, (4) goon until death
@gooncultist The key is that llama.cpp is a C++ implementation that's distributed as a binary, so you don't need a bunch of pytorch bullshit. Macs have integrated GPUs these days, so there's no separate driver, and LLM folks seem to also be Mac folks, so support is there by default.