Pedro Moser

[Qwen Developers @QwenDevs]

@ddalcu We promise this 27B comes with a whole new level of capability!

[Pedro Moser @outrotuiter]

@angelus513 @TeksEdge Measured with llama.cpp timings, temperature 0, fixed 384-token outputs:
baseline: 16.85 tok/s
repetitive/code copy with ngram-mod: 47.21 tok/s — 2.80×, 86.7% draft acceptance
summarization: 16.75 tok/s, no regression
fresh writing: 17.07 tok/s, effectively unchanged

[Pedro Moser @outrotuiter]

@angelus513 @TeksEdge --spec-type ngram-mod
--spec-ngram-mod-n-match 32
--spec-ngram-mod-n-min 48
--spec-ngram-mod-n-max 64

these are very helpful for decode speed in repetitive output

[Pedro Moser @outrotuiter]

@angelus513 @TeksEdge rtx 5080 + 128 ddr5 6000 cl40 ryzen 9800x3d
ds4flash iq3_XXS
kv q8 256k
flash-attn on
fit-target 2000
ubatch-size and batch-size 2048 (this is important, major prefill boost)
threads 8 (sweet spot for my cpu)
--load-mode none (major prefill boost)

[Pedro Moser @outrotuiter]

@iamlukethedev Link?

[Pedro Moser @outrotuiter]

@appariciojunior É patético. Eu cancelei meu 5x. Agora vou de IA local e algumas chamas API para funções específicas. Deepseek4flash e laguna s 2.1 mudaram o jogo

[Pedro Moser @outrotuiter]

@HermesWatcher Deepseek v4 flash q3 local here! But qwen 3.6 35ba3b comes in when I need more speed

[Pedro Moser @outrotuiter]

@iamlukethedev vc tá criando algum tipo de grupo? onde?

[Pedro Moser @outrotuiter]

Ta louco! Deepseek v4 flash local + Hermes é uma coisa de louco! Cancelei minhas assinatura tudo azar

[Pedro Moser @outrotuiter]

@gnukeith it's amazing because i've upgraded to 128gb of ram last night. i woke up testing laguna s 2.1 and now i can't sleep because DS4flash is too damn good <3

[Pedro Moser @outrotuiter]

@0xCVYH credo que delicia

[Pedro Moser @outrotuiter]

@TechMDAI damn, i'm loving it so much on a 5080 + 128gb ddr5. i can't even image how great it must feel on 4x GB10 <3

[Pedro Moser @outrotuiter]

@MichaelHutu i've running a q3 on a 5080 + 128gb of ddr5. i'm getting ~18t/s. i'm 100% in love. cant stop working with it, probably wont sleep tonight lol. just canceled my anthropic max x5 subscription.

[Pedro Moser @outrotuiter]

@evertjr Meu plano 5x foi 50% em um dia de trabalho com opus 5.

[Pedro Moser @outrotuiter]

@0xCVYH na moral, esse setup não faz nem sentido kkkk pessoal tem que aceitar que o química3 não é um modelo local e deu.

[Pedro Moser @outrotuiter]

@eric_alcaide To me it’s clear you didn’t! The XS outperformed all similar models in my agentic coding workflow, despite scoring less in benchmarks. I’m buying ram to run the S version as my daily driver.

[Pedro Moser @outrotuiter]

@superalesha The question is: why are you running a 3.6gb model if you have 4 3090s? Hahahaha

[Pedro Moser @outrotuiter]

@TeksEdge @nvidia @wccftech windows 🤮