I ran a 35-billion parameter AI agent on a $600 Mac mini.
Specs: M4 Mac-Mini 16GB RAM
The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second.
On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster.
No cloud. No API keys. $0/month.
Here's what it can do ๐งต
@TheFlowHorse@breakoutprop Itโs arguably the best prop around. Wish it tied into insilicoโฆthe current terminal is kinda horrendous. Hopefully with the kraken partnership itll improve. Fees get big fast too unfortunately. But otherwise great