Chintan Vadhadiya

[Chintan Vadhadiya @marutichintan]

@NullBurn I got the prefill but not decode

[vr8vr8 @vr8vr8]

Qwen3.8-Flash-Next tp=2‼️
DS4.1-Flash looked like a miracle running at 200+ now we have similar speed for Qwen3.8 locally 177tok/s‼️
🚀 Peak c=1 @ 177 tok/s
📊 Average c=1 @ 93 tok/s
🏭 Peak c=64 @ 1,174 tok/s
⚙️ Prefill 3,939 tok/s
💻 TTFT 0.36s
⚡️ https://t.co/vzLhro5IXC

[Chintan Vadhadiya @marutichintan]

@mmastrac Tp 4 switchless ?

[Chintan Vadhadiya @marutichintan]

@mmastrac it switchless ?

[Chintan Vadhadiya @marutichintan]

@JohnWmm I am using it, it really good

[Chintan Vadhadiya @marutichintan]

@uaghazadae You can use Buffer

[Ash Hart @ashxhart]

I just killed Time to First Token. 🚀

Fresh sessions kept making my model reread the same opening prompt/tokens. So I precomputed them once and saved the state.

My TTFT dropped from 19s to 0.3s, a 63× speedup with zero quality loss.

As usual, I will be doing a PR for oMLX.

https://t.co/HQudCXL3nR

Video in the comments 👇🏼

[Zbigniew Majewski @majewskizby]

More DS4.1 TP4 improvements on 4x DGX Spark 🚀

- Prose decode: 85 tok/s at c1 (was 69)
- Boot: ~3 min
- Prefill: ~4.8k tok/s at 32k–128k, 4.3k at 262k
- KV cache: ~6.5M tokens, 1M context
- Same weights, same quality (qeval unchanged)

https://t.co/tYFYjyCzhY

[Chintan Vadhadiya @marutichintan]

@majewskizby Its switchless ?

[Chintan Vadhadiya @marutichintan]

@vr8vr8 Just testing now

[Chintan Vadhadiya @marutichintan]

@MiaAI_lab Do something for dual

[avrl ☘ @avrldotdev]

you'll never be unemployed if you master inference engineering, but you need to be unemployed to master inference engineering.

[Chintan Vadhadiya @marutichintan]

@Tech2Wild @u1tra_instinct Switchless ?

[Chintan Vadhadiya @marutichintan]

@vr8vr8 It not error, it forgets calling

[Chintan Vadhadiya @marutichintan]

@basewtf @MiaAI_lab @imost concurrent prompt are not working, we need to enable concurrent then it drop token/sec

[Chintan Vadhadiya @marutichintan]

@XadenRyan @ashxhart @exolabs I have the same setup, not able to do that

[Chintan Vadhadiya @marutichintan]

@exolabs Waiting to connect dgx with mac ultra

[Mia @MiaAI_lab]

DeepSeek v4 Flash Vision Exp is now open-weight! 🔥

https://t.co/lDAWSPWksq

[Chintan Vadhadiya @marutichintan]

@Tech2Wild Which switch you to connect 4 dgx