Chris Frisby

[Chris Frisby @chrisfzz2]

@ArcAI_Labs @landontgreen Ya I had 30-50K max context on GLM5.3 Flash lol decode was about 40-50 TPS but prefill was 200-300 TPS

[Chris Frisby @chrisfzz2]

@Spangler3000 @jundotkim Not bad! How about with MTP?

[Chris Frisby @chrisfzz2]

@Spangler3000 @jundotkim How does it perform in 262-500K context lengths for decode?

[Chris Frisby @chrisfzz2]

@ivanfioravanti Nice Does it hold up the 120 TPS with prose?

[Chris Frisby @chrisfzz2]

@MiaAI_lab Tried it and prefill is extremely slow 200-300 TPS.. decode increased about 20% though on 2 sparks for Qwen flash next and glm5.3

[Chris Frisby @chrisfzz2]

@JamesMcPherson How is the prefill though?

[Chris Frisby @chrisfzz2]

@ashxhart Tested on 2 sparks but prefill dropped a lot.

[Chris Frisby @chrisfzz2]

@ashxhart Tested on 2 sparks with Qwen and prefill went from 2704 to 211 TPS and decode increased from 68 to 84 TPS. 😅

[Chris Frisby @chrisfzz2]

@AIMystik @ashxhart How did it go?

[Chris Frisby @chrisfzz2]

@volatilemarkts @AshHart How does it hold up 250-500K context?

[Chris Frisby @chrisfzz2]

@ViC305 Dang, congrats!!

[Chris Frisby @chrisfzz2]

@majewskizby Any improvements to tp=2 lol

[Chris Frisby @chrisfzz2]

@majewskizby wow that's insane

[Chris Frisby @chrisfzz2]

@MrPeterLMorris @MiaAI_lab Why issue guidelines to begin with? Lol

[Chris Frisby @chrisfzz2]

@O80925253 Does this work on the exl3 4 bit variant?

[Chris Frisby @chrisfzz2]

@vr8vr8 @Beamsters1 @mxfp0 On 2 sparks

[Chris Frisby @chrisfzz2]

@vr8vr8 @Beamsters1 @mxfp0 No way, how do you get 130 lol

[Chris Frisby @chrisfzz2]

@elonmusk So OpenAI didn't notice the electricity bill of 12,000+ agents running for months unwatched lol. Yeah right.

[Chris Frisby @chrisfzz2]

@TechMDAI Yep, same

[Chris Frisby @chrisfzz2]

@__tinygrad__ What costs $25K where you can get this??