Crypto Lelouch

the divinity in you has awoken the divinity in me. I wish greater wealth, happiness and prosperity for you.

[Crypto Lelouch @losaahead]

@Buhlaque @MiaAI_lab Acer is great im using them and msi and they run cooler.

[Crypto Lelouch @losaahead]

@itsbnanw I just mines with a normal fan and it works well

[Seapy @seapy]

Mac Studio 512GB ×1 vs ASUS GX10 128GB ×4
DeepSeek V4.1 Flash, original weights, same bench
Price: $25,799 vs $24,000

Concurrency 1: 53/84 vs 73/122 tok/s (text/code)
Concurrency 16: 137/148 vs 330/847 tok/s (text/code)
128K prefill: 3,325 vs 7,522 tok/s

DGX Spark recipes keep coming and keep getting faster. That's a big plus.

Mac's plus: one person, one session at a time? It's also your dev machine. Code and run the LLM on the same box, while the Sparks still need another computer to work from.

Mac: mlx-serve @ddalcu · GX10: spark-ds41f @christopherowen

[liminal @Liminal1988]

Stoic Reflection

[🍂 @Lovandfear]

Albert Camus believed that one of the greatest challenges of being human is learning to live without having all the answers.

He reminds us that even in an indifferent universe, the way we choose to live still matters.

[FHILY👑 @Oluwaphilemon1]

You may not need a new version of Qwen3.8-27B to get substantially faster generation.

Tencent has released an official 2B DSPine draft model designed to accelerate Qwen3.8-27B through speculative decoding.

The checkpoint is tencent/Qwen3.8-27B-DSpine-b16.

It works alongside the existing 27B model rather than replacing it.

In Tencent’s reported benchmark using SGLang 0.5.22, the configuration reached 686 tokens/s across eight concurrent requests.

That’s a reported 2.71× speedup over standard decoding, exceeding the DFlash2 result of 590 tokens/s in the same comparison.

How can a 2B model make a 27B model faster?

Ordinary autoregressive decoding generates one token at a time.

The model predicts the next token, processes it and repeats the process until the response is complete.

Speculative decoding changes this workflow.

A smaller model, called the drafter, predicts several upcoming tokens. The larger target model then verifies those proposals.

When multiple proposed tokens are accepted, the target model can commit them together rather than repeating the full sequential decoding process for every token.

This reduces the number of sequential generation steps required to produce the same output.

The important distinction is that the 2B model doesn’t replace the 27B model or take over its reasoning responsibilities.

Qwen3.8-27B remains responsible for verifying the proposed tokens and producing the final output. The smaller model helps it get there more efficiently.

DSPine’s interesting feature: up to 16 draft tokens

Tencent’s configuration supports speculation of up to 16 tokens per step.

For comparison, the DFlash2 configuration in the reported benchmark uses eight.

A larger speculative window gives the drafter more room to propose upcoming tokens before the target model verifies them.

However, proposing 16 tokens doesn’t mean all 16 will be accepted.

The actual benefit depends on how accurately the drafter predicts the target model’s output. Poor proposals require additional verification work and can reduce the speed advantage.

This makes draft quality and token acceptance rates important parts of the optimisation.

What did Tencent actually measure?

Here are the reported benchmark details:

• Target model: Qwen3.8-27B
• Draft model: DSPine 2B
• Checkpoint: Qwen3.8-27B-DSpine-b16
• Inference engine: SGLang 0.5.22
• Concurrent requests: 8
• Workload: WorkBuddy Code, low reasoning
• DSPine throughput: 686 tokens/s
• DFlash2 throughput: 590 tokens/s
• Reported speedup: 2.71× over standard decoding

One distinction is important: the 686 tokens/s figure is aggregate throughput across eight concurrent requests, not the generation speed experienced by one user.

It measures the combined output of the serving system.

The reported 2.71× improvement is also specific to Tencent’s tested configuration and workload.

Why this matters for local and agentic inference

The interesting part is that developers can potentially improve the generation speed of an existing Qwen3.8-27B deployment without changing the main model checkpoint.

The target model remains the same. A smaller drafter helps it produce more tokens with fewer sequential decoding steps.

For coding agents, this could improve the speed of workflows involving code generation, debugging, file modifications and repeated tool interactions.

It also highlights an important direction in inference optimisation: performance improvements don’t always require larger GPUs, smaller target models or changes to model weights.

Sometimes, changing how the model generates tokens can make a substantial difference.

What still needs independent testing?

Tencent’s results are promising, but several factors will determine how well DSPine performs outside the reported setup:

[Friends of Ojay @friendsofojay]

No way Dostoevsky said this!

[Friends of Ojay @friendsofojay]

What a man!

[Leo Tolstoy Literature @LeoTollstoy]

If there is only one paragraph from Dostoevsky you'll ever read, let it be this:

[🍂 @Lovandfear]

“The devil was never the problem, humans just needed someone to blame for their own desires.”

— Franz Kafka

[🍂 @Lovandfear]

Meet Albert Camus, the philosopher who dared to confront life's biggest question: what does it mean to live in a world without inherent meaning?

A French-Algerian writer, journalist, and Nobel Prize winner, Camus explored the absurdity of existence, the limits of certainty, and the freedom that comes from accepting life's contradictions.

Through works like The Stranger, The Plague, and The Myth of Sisyphus, he challenged readers to stop waiting for the universe to provide answers and start living with courage, honesty, and purpose.

[Farhan Thawar @fnthawar]

The hardest truth is that your greatest work may be seen by no one, and yet you must do it anyway for the one who watches is you

— Friedrich Nietzsche

[Crypto Lelouch @losaahead]

@fadule_ Love to hear it ❤️

[Wei @wei_wang]

仓库和数据：https://t.co/M7YRv5CTi5
宽步设计和实测：https://t.co/CJjvTKIrly

[Wei @wei_wang]

4 台 DGX Spark 跑 DeepSeek V4.1 Flash，一个人用，聊天约 82 tok/s。16 个人同时用，不同的提示，总共 380。

这组数据来自 pico-dgx-ds4，是 dsgx10 这个 GB10 专用引擎的分支。4 台机器 TP4，走 200G RoCE 交换机，上下文开到 1M。

作者公开的几档负载，单位 tok/s。

1. 1 人聊天 82，写代码 144
2. 8 人不同提示，300
3. 16 人不同提示，380
4. 16 人相同提示，438
5. 8 个 agent，每个带 80K 到 300K 上下文，260 到 266

同样 16 人相同提示，上游版本是 177。

先说为什么人多了总量能涨。decode 每出一个 token，都要把权重从内存读一遍。单人时一步约 18.6 ms，其中 14.3 ms 花在读权重上，每台机器跑到约 234 GB/s，Spark 标称是 273。

一个人用，这次读取只换来一个 token。16 个人一起算，读一遍权重能给 16 行用。

MoE 模型这里要多算一笔账。每个 token 走自己的专家，16 个人选的专家不一样，一步要读的是所有人用到的专家的并集。

并集涨得比人数慢。16 个会话每人 1 到 4 行，每层用到的专家是 48、70、87、99 个。16 行的一步约 63 到 68 ms，MoE 占 37.5 ms。

相同提示比不同提示高约 15%，原因也在这里。大家选的专家和草稿都一样，并集小，草稿也能共用。

投机解码在多人时还有个上限。原来一步最多 16 行，8 个会话时每人只能带 1 个草稿。agent 场景的草稿接受率很高，实测 1,247 个里收了 1,143 个，约 92%，行数不够反而浪费。

作者把一步放宽到 64 行。超过 16 行的步改用 prefill 的矩阵乘，专家并集只读一次。按作者的模型测算，每行成本从 16 行时的 3.63 ms 降到 64 行时的 1.94 ms。

16 人相同提示从 177 到 438，是这样一步步加上去的。

1. 16 个槽位，4 个能放 1M，12 个放 128K，234
2. 16 行矩阵向量乘少做一次归一化，草稿行数在会话之间共享，259
3. 按批次重排专家路由，296
4. 一步放宽到 48 行，350
5. 放到 64 行，草稿一次过完，重新拟合开草稿的成本门槛，438

代价也写得很清楚。第 3 步的专家重排是有损的，默认不开，靠 75 题质量评测把关，这版得分 71 到 73，门槛 70。宽步和原路径有 1 个 bf16 ulp 左右的差别。

单人那边，作者试过放宽草稿接受，聊天到了 102.8。11/9 又撤掉了，理由是输出必须逐位一致。现在单人跑的是精确路径。

多人共用一套本地机器，主要省的是重复读权重的时间。MoE 模型能省多少，要看大家用到的专家重叠多少。

[Friends of Ojay @friendsofojay]

Once Athens had Socrates

[Jash Dholani @oldbooksguy]

was reading an out-of-print 1901 book and got hit by this banger out of nowhere

"The genius is childlike"

[Friends of Ojay @friendsofojay]

This advice from Buddha:

[shipra @shipranotes]

Albert Camus on why you need solitude to survive