@deeprelayai In contrast, latency is quite good with shorter contexts.
It seems that TTFT degrades significantly as the context grows. I hope you can look into long-context prefill performance and optimize it. It would make a big difference to the overall user experience.
@deeprelayai DeepSeek-V4.1-Flash.
After using it for a while, I’ve noticed that the high time-to-first-token (TTFT) latency mainly occurs with long contexts. For example, with a context of around 80K tokens, the first token can take about 30 seconds to arrive.