llmfit. Useful tool that probes hardware and tells you exactly which LLMs will actually run.
- handles MoE expert offloading, picks the best quantization for your RAM, estimates tokens/sec before you even pull the weights.
Essential for local dev.
https://t.co/qn8PmqQNbV