xM_My

Local LLM

[xM_My @NullBurn]

Correction: I found my earlier 100+ tokens/s result was inflated by an API proxy buffering tokens and delivering them in bursts, which distorted SparkDash’s client-side decode measurement.

[xM_My @NullBurn]

Running GLM-5.3-Flash on 2× DGX Spark with NVIDIA Flare ✨

- 1M context window, 2.32M-token shared KV pool
- Prose decode, single stream: 117.2 tok/s
- Prose decode, 2 concurrent: 167.4 tok/s aggregate
- Prose decode, 4 concurrent: 215.8 tok/s aggregate
- 32K prefill: 2,327.9 tok/s
- Available memory after loading: ~13 GiB on head /worker each

Checkpoint: GLM-5.3-Flash-NVFP4-MXFP8-CSF-QAD

https://t.co/3VizjM0dUK

[xM_My @NullBurn]

@RipperdocMD Both at max thinking level? In my case, DeepSeek V4.1 Flash 1017 always thinking too much more than GLM 5.3 Flash.

[xM_My @NullBurn]

@Confetti1279 @ivanfioravanti local-inference-lab/GLM-5.3-Flash-NVFP4-MXFP8-CSF-QAD

[xM_My @NullBurn]

@J37270031

[xM_My @NullBurn]

Running GLM-5.3-Flash on 2× DGX Spark with NVIDIA Flare ✨

- 1M context window, 2.32M-token shared KV pool
- Prose decode, single stream: 117.2 tok/s
- Prose decode, 2 concurrent: 167.4 tok/s aggregate
- Prose decode, 4 concurrent: 215.8 tok/s aggregate
- 32K prefill: 2,327.9 tok/s
- Available memory after loading: ~13 GiB on head /worker each

Checkpoint: GLM-5.3-Flash-NVFP4-MXFP8-CSF-QAD

https://t.co/3VizjM0dUK

[Mia @MiaAI_lab]

NVIDIA just released Flare, a local inference server built for DGX Spark and RTX Blackwell GPUs

- Built for long chats, agents and tool calls
- Serves from Rust, no Python
- Plans memory up front
- FP8 KV cache by default

Interesting timing... right around when DeepSeek open-sourced their inference engine.

Link below.

[xM_My @NullBurn]

prefill 2300tps @ 32k

[xM_My @NullBurn]

@MiaAI_lab yes!! NVIDIA flare !

[xM_My @NullBurn]

@0xIlyy @FreeofspeechUSA @autodidacc_fyi 请问瓶颈主要是在哪呢？ 能否跑nvidia/GLM-5.3-Flash-FP4-Local

[xM_My @NullBurn]

@Reederey which recipe?

[xM_My @NullBurn]

@MiaAI_lab no, sacrifice 5-10% decode speed over maximum of 5-10% quality

[xM_My @NullBurn]

@naawwwal @ProEvilz @X @ego_agent 用了jev吗

[xM_My @NullBurn]

@mxfp0 need system prompt asking him return commentries

[xM_My @NullBurn]

@uzairansar 移除了webui吗？

[xM_My @NullBurn]

@benwvlshvm how about it compare to glm5.3 flash

[xM_My @NullBurn]

@uzairansar 期待～

[xM_My @NullBurn]

@mxfp0 @Zai_org @BalatroGame Which harness and tool did you use for control the computer?

[xM_My @NullBurn]

@satsuki2694 try this recipe https://t.co/Umr30MHynG

[xM_My @NullBurn]

@TeksEdge how about Ryzen AI Max+ PRO 495 vs 2 dgx sparks?

[xM_My @NullBurn]

@wlzh @NullBurn 看看这个具体能做什么？