🚨 BREAKING:
#2 on @hackernews!! engineers figured out how to serve GLM 5.2 on @AMD MI355X at 2626 tok/s/node and 213 tok/s single stream at over 2x lower cost than B200
‼️~80% of B200 throughput at over 2x lower cost
@0xSero This sounds great, I would love to watch and support a US college LocalAI tour. I was going to list reasons for why this would be great but I think the reasons are countless.
@0xSero Another key factor I could email about is model control, fine tuning, and the fact that the downtimes of servers in Texas won’t effect students in Pennsylvania.
@0xSero I’m up in Erie, PA if you’re up for it I’ll email in and we will get a date setup the presidents great at Gannon. They’ve been working towards environmental goals too so this could be a big movement.