First look at Tencent’s Hy4-flash-FP8 (220B total / 12B active MoE) running at native checkpoint precision (W8A16) on 2× DGX Spark:
• 98.3 tok/s across 8 distinct prompts (89.0 tok/s even without speculation)
• 43.6 tok/s single-stream (144 ms TTFT)
• ~2,000 tok/s cold prefill all the way to 32K (1,983 @ 8K, 1,969 @ 32K)
Scale to a 4× Spark switchless 200GbE ring: 58.0 tok/s single-stream (93 ms TTFT) and 123.5 tok/s across 8 distinct streams, with 42–57 GiB unified memory still free per box.
To fit full precision on 2× Sparks, the 42-layer backbone + built-in 5-layer DFly2 drafter stay hot in unified memory (80 GiB/box) while the 61.5 GiB Engram lookup tables stream straight off NVMe via async io_uring O_DIRECT at 96–100% of RAM speed.
Built on @ashxhart’s TensorFold with custom GB10 CUDA kernels. Wiring up the vision encoder and final KV pool tuning next, open recipe + kernels dropping later today.
@asmDevsit@Picomoorhead After a kill -9 the engine is serving again in ~50 s (process restart + weights). Reading a session back from NVMe then takes ~0.1 s: a 180K-token revisit 0.096 s vs 19.4 s to prefill it cold (36K: 0.087 vs 3.99 s)
DeepSeek V4.1 Flash on 4× DGX Spark: one shared KV pool, and sessions on NVMe that survive kill -9.
Fork of @Picomoorhead's pico-dgx-ds4 (built on Andy Shu's dsgb10):
• a live session reserves 5.9 MiB, was up to 895
• 128 GiB disk tier per box
• exact: golden hashes 8/8 🧵
@orenoa7 Great comparison! If you run it again, we would love our fork in it: pico-dgx-ds4 with one shared KV pool and crash-proof sessions on NVMe, replies exact at any concurrency. Happy to help set it up: https://t.co/PFmYAaA1x8
Our fork of this: one shared KV pool (a live session reserves 5.9 MiB, was up to 895) and sessions on NVMe that survive kill -9. v0.1.0:
https://t.co/xZ1GNDxaSP
How: coding agents wrote most of it, several running DeepSeek V4.1 Flash on this same cluster, inside test windows we designed and gated. Results and method:
https://t.co/3pjcdX8LFg