Mads Henrichsen

[Mads Henrichsen @MGHenrichsen]

@ChiragAsarpota @mfranz_on 262k https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@apeiron_spx @mfranz_on https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@theyknowitstrue @analogalok @spiritbuun @blan sure

[Mads Henrichsen @MGHenrichsen]

@superalesha Except it can run at 140 tps with a single 3090 :) https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@MarshallJGould try +1100 tps on a single 3090 https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@LottoLabs https://t.co/i1ALoXckr6 try 138 on a single rtx 3090.

[Mads Henrichsen @MGHenrichsen]

@malikwas1f lol try 1000 tps https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@theyknowitstrue Benchmarks in the repo too.

[Mads Henrichsen @MGHenrichsen]

I made Qwen3.8-27B run with up to 381 TPS on a single request.

Up to 8 times faster than the default vLLM implementation.

https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@mstefanec Going to fix the short context in the fast mode first then look at increasing prefill.

[Mads Henrichsen @MGHenrichsen]

I made the fastest inference stack for Qwen3.8-27b on a RTX 3090.

Up to 195k context, 82 tps single request, 672 tps peak 64 concurrent.

https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

This is on a RTX 3090 power limited to 250w.

[Mads Henrichsen @MGHenrichsen]

@malikwas1f @Alibaba_Qwen 381 tps on a single 3090: https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@TeksEdge Thanks for posting. It’s my optimizations. Let me know if you have any questions.

[Mads Henrichsen @MGHenrichsen]

@mfranz_on https://t.co/i1ALoXckr6 124 tps single request

[Mads Henrichsen @MGHenrichsen]

@sudoingX @atomic_chat_hq https://t.co/i1ALoXckr6 124 tps single request

[Mads Henrichsen @MGHenrichsen]

@0xSero We can do much better than this :)
https://t.co/UPuXmOHqdl

[Mads Henrichsen @MGHenrichsen]

@XReyRobert not my model, just my optimizations

[Mads Henrichsen @MGHenrichsen]

@MiaAI_lab https://t.co/i1ALoXckr6

[Mads Henrichsen @MGHenrichsen]

@XReyRobert So little love indeed. New version coming soon with 120 tps single request.

[Mads Henrichsen @MGHenrichsen]

@burkeholland Use my inference lib and get 672 tps peak :)
https://t.co/i1ALoXckr6