Merlin 3.0.0 is out. My Flash-Next kit is now a proper vLLM fork for RTX 30/40-series & More: ready-made recipes for seven checkpoints, one install, one command.
Flash-Next on 4× RTX 3090, no NVLink, 220 W:
• 184 tok/s decode at the full 262K context
• 6,051 tok/s prefill on a 100K prompt (+12 %)
• 925K-token KV pool, 3 full 262K sessions resident
• Token embedding tables in host RAM
• 8 agents at once: 638 tok/s
we seriously need to have a conversation about how the overwhelming prevalence of meme culture has made people insensitive to serious tragedies and only see them as an opportunity to make a joke rather than for the reality of the situation and those in it