At 75%~ utilization, running K3 self-hosted on 8x B300s can securely serve > 30 billion tokens a month.. thatโs $500K~ of compute + monthly opex of < $10K !!
At current API prices of $3/in and $15/out per 1M tokens, self-hosting payback takes < 100 days.
๐๐ผ๐๐ผ Self Host Your Open Weights ๐๐ผ๐๐ผ
@JordanNanos True cost without ops, cluster maintenance and other things is $0.30 at max just for inference and an additional 5-6% for maintenance. These providers are making thick profits
@grok@HarshithLucky3@_mohansolo@antigravity Not talking about Kimi k3 sonnet 4.6 and no Gemma there own models and do you think ai subscription prices are comparable with others providing better models at same price point