Marc Seal

Founder of Aevonix | @get_hydrahost Sovereign Strategy Partner | Between El Salvador, New York & Switzerland | Longevity and Access to Intelligence for All

[llmwish.open @ciprianveg]

**This is the 2.8T-parameter Kimi K3 running at ~30 tok/s on 16× NVIDIA DGX Spark.**

Not a synthetic “it boots” test.

👇 **Actually generating. Usable speed.**

A few weeks ago, the question was:

*Can a model this big even run properly on a cluster of Sparks?*

Then:

*Can we make it fast enough to actually use? What about for agentic frameworks?*

I think we're well past that question now.

With the new **V5**, we're seeing roughly another **20% speed-up**, plus a big improvement in concurrency:

⚡ **C1: 29.8 tok/s**
⚡ **C2: 42 tok/s aggregate**
⚡ **C4: 58 tok/s aggregate**
⚡ **C8: 87.1 tok/s aggregate**
⚡ **136 tok/s peak at C8**

And long context remains usable:

→ ~20 tok/s @ 300K context

So **Kimi K3 on DGX Spark is no longer a “yes, it boots” experiment.**

It's becoming a genuinely usable platform for **agentic coding, long-running agents and concurrent workloads**.

Getting here required work across almost the entire stack:

→ GB10 / SM121-specific vLLM work
→ Kimi-specific fixes and patches
→ optimized MLA / CuTe kernels
→ DSpark speculative decoding nst 3/6 fusioning
→ FlashInfer / Triton / CUDA work
→ KV-cache/ DCP + long-context fixes
→ NCCL / RoCE tuning
→ MoE fusion + nopad — projection-transport fusion + per-rank slice legality (VLLM_K3_MARLIN_NOPAD=1) and many other small fixes, from open source community and from us.

There wasn't one magic flag.

It was dozens of improvements stacked together until a **2.8 trillion parameter model on 16 tiny GB10 boxes became something I actually want to use.**

Code, image, recipes and patches:

https://t.co/HkMfJYLqhF

**Pull it. Run it. Break it. Improve it.**

I'm pretty sure we still haven't found the ceiling.

If somebody tells you Kimi K3 on DGX Spark is just a proof of concept…

**show them this video. 🙂**

https://t.co/oIf2z4v4W6

@Kimi_Moonshot @NVIDIAAIDev @vllm_project

[雨夹雪❄️ @mizorewww]

介绍比Jev快50倍，在你设备上跑的laya-mlx!

只在你的设备上占用最高1G内存

Laya是一个开源的类似于Jev的,基于文本输出概率的分类系统

我将其移植到MLX,并且做了一些性能优化!

视频中就是这个模型在我的本地M3Max上玩贪吃蛇

这个模型能够以每秒决策60次的速度玩贪吃蛇！

https://t.co/31KGUiNunb

[Mia @MiaAI_lab]

Qwen Image 2.1 open weights are now LIVE 🔥

https://t.co/Ug5OCfIKEk

[Mia @MiaAI_lab]

This is going to change everything 🚀

DGX Sparks and Mac Studios working together for optimal performance.

We are so early.

[Marc Seal @Kurcyde]

The entire plan is to slow the public down so they are on unequal footing.

Don’t fall for it, support progress and support open access to super intelligence.

If we don’t and open access loses, the resulting societal gap will dwarf that of the poor vs rich class segregation.

[Joscha Bach @Plinz]

Nobody is planning to stop or pause AI. Big tech companies and governments will be improving and using frontier AI as a matter of course. But a lot of people are aiming to keep capable AI models out of the hands of the general public.

[Ash Hart @ashxhart]

MCDMA | Metal CUDA Direct Memory Access 🚀

If you have a Spark and an Apple Silicon Mac, MCDMA gives you a direct RDMA path between CUDA memory and Metal-side unified memory over USB-C.

Registered memory, rkeys, one-sided READ/WRITE, two-sided SEND/RECV with credit flow control. Same verbs both ways, no master/slave. The Mac writes straight into CUDA-mapped memory on the Spark, and the Spark writes straight back into Mac memory.

My setup takes it a little further:
Spark 1 ⇄ CX7 ⇄ Spark 2 (prompt processing)

Spark 1 ⇄ USB-C ⇄ Mac Studio (Decode)
Spark 2 ⇄ USB-C ⇄ Mac Studio (Decode)

Two independent MCDMA USBC links, so the Studio isn't stuck behind one cable; both Sparks move data concurrently, and it writes results back into either.

Measured, every byte delivery verified:

• 939 MB/s single link
• 1.80 GB/s Mac → both Sparks, concurrent
• 1.25 GB/s both Sparks → Mac, concurrent
• 24 µs round-trip, 41k msg/s small-message

One Spark + One Mac works. Two is just how I'm using it: DeepSeek prompt processing across the Sparks, decode on the Studio.

Benchmarks, tests, Open Source, and write-up this week.

@NVIDIARTXSpark @NVIDIAAI @NaderLikeLadder @msharmavikram There’s still a lot of performance headroom here. If the currently locked USB4 controller can be allowed to train at full capability, I’d love to test how far we can push this. Please check your DMs.

[Volatile Markets @volatilemarkts]

IT WORKS!

For a year, anyone with DGX Sparks and Mac Studios has lived with the same problem: the NVIDIA boxes are fast at reading, the Apple boxes are fast at answering, and they can't share a thought. Two islands. A 10 GbE cable between them.

Earlier this month we published pd-bridge (https://t.co/GVsHqq9MHC): prefill/decode disaggregation across NVIDIA and Apple silicon. Spark 1 and Spark 2 run the prefill, a Mac Studio runs the decode, and the KV cache crosses between them — reconstructed on the Mac bit‑exact (313/313 arrays) so the decoder sees a normal prefix‑cache hit and never knows a bridge exists. Up to 3.7× faster than a Mac alone at 241K tokens, over plain 10 GbE HTTP.

That was one‑way, one turn, one Mac.

Today it's a loop!

The question we set out to answer:

Can a single conversation circulate across four machines — two DGX Sparks, two Mac Studios — turn after turn, each doing only what it's built for, and beat a single machine doing the whole job?

How it works:

• Spark 1 + Spark 2 (vLLM TP2) — Prefill. Read the conversation, compute the attention state. Blackwell compute: ~1,100 tok/s vs ~400 on a Mac Studio.

• Mac Studio A — Gateway. The only Studio with a Mellanox ConnectX‑4 on the MikroTik fabric (via a Thunderbolt 5 enclosure). Receives the Sparks' cache over RDMA — kernel bypass, no TCP, no files — using @ashxhart MCDMA driver ( https://t.co/C6L7t2gfAo) and maps it into oMLX cache blocks. First time that driver has run on a ConnectX‑4. First time through a mikrotik 812 switch. 6 µs latency, down from ~400! This is the key and we cannot stress enough how appreciative of his work, efforts, and willingness to give to the open source community!

• Mac Studio B (512 GB) — Memory. Holds the entire conversation's cache, receives each new piece from A over Thunderbolt 5 RDMA, and generates the reply. Never re‑reads the history, because it already has it.

• MikroTik CRS812 + TB5 mesh — Two kernel‑bypass RDMA fabrics carrying attention state between three machines.

The reply becomes part of the next message and goes back to the Sparks. Inference, back and forth, across two kinds of silicon. NVIDIA computes what Apple speaks.

The big wins:

• Up to 4.2× faster replies than a Mac Studio alone. Alone, Mac Studio B sits at a flat ~31 s per message because it re‑prefills the entire history every time. Fed by the Sparks: .7 s at 31K tokens, 7.4 s at 47K, 10.5 s at 63K. The longer the conversation, the more the split pays.

• The Mac never prefills. Cached tokens on Mac Studio B: 30,720 → 47,104 → 61,440. Every block computed on Blackwell, consumed on Metal.

• Network transport is a rounding error. Spark → Mac Studio A over RDMA through the MikroTik: 0.3–0.5 s. Mac Studio A → B over TB5: ~0.05 s. Under 1 second of a ~8‑second reply.

(Next: routing the cache back to the Sparks so they only prefill what's new — that flattens the whole loop.)

Who made this possible:

@ashxhart — MCDMA. RDMA verbs on macOS.
Nothing here works without it! @Apple @NVIDIAAI @mikrotik_com (https://t.co/C6L7t2gfAo)

@b_ostrov — MelonDMA, and the KV‑block‑over‑RDMA ideas we built on, and because I love that Ben is using M1-M2 machines and making this backward compatible for anyone/everyone with a card and imagination..

@MiaAI_lab — the model builds we run, because you already know she's going to be the queen of the heterogeneous model scene!

Why we're posting:

This is a basement, not a lab. We're not selling anything: compute where compute is cheap, memory where memory is cheap: RDMA in between.

What if we don't stop here..? Open Source Must Win!

[Mia @MiaAI_lab]

Official DeepSeek v4.1 Flash NVFP4 from @NVIDIAAI 🔥

I will be testing it vs native weights and my EXL3 quant.

https://t.co/wmOGoOTOaG

[Mia @MiaAI_lab]

I'll be working with @ashxhart to refine cross-architecture local AI recipes.

Goal is DGX Sparks for prompt processing, Apple Silicon for decode, MCDMA moving the tensors.

[Marc Seal @Kurcyde]

@MiaAI_lab Can’t DM you but would love to coordinate and see if I can support you

[Marc Seal @Kurcyde]

@MiaAI_lab This was after I set them up: https://t.co/c8a5rUNQVU

[Marc Seal @Kurcyde]

4x 4090s across 2 workstations
4x H100 NVL server
1x GH200
and now….
16x DGX Spark Cluster

All in my basement

Let’s try and get DeepSeek 4 Pro running in the sparks, report back soon

@nvidia @nvidiadeveloper Who can I talk to about some support on this project?

[Marc Seal @Kurcyde]

@MiaAI_lab yup, it’s from my localllama reddit post when I setup the first 16 Sparks. That’s my stand for my record player to the left 😂

Now I have 24 in the cluster and 12 more waiting to be setup

[Marc Seal @Kurcyde]

@MiaAI_lab Ok… but what about 36 Sparks?

[Marc Seal @Kurcyde]

@MiaAI_lab that’s me! hahahha 😂

[Marc Seal @Kurcyde]

@iAjittiwari Our own system, check out

https://t.co/BBNjFeaIJs

[Marc Seal @Kurcyde]

I built a local AI Agent called AEVA running on our open source autonomy, memory and learning system built for use with Hermes agents called Protagine.

You can now battle her in Pokemon to help me test learning and memory retention across sessions.

https://t.co/io7CVuDLoN

[Marc Seal @Kurcyde]

You can now battle AEVA, a completely sovereign and local AI agent running on Protagine, our open source autonomy, memory, and learning engine for Hermes agents!

Help us test her trainer skills here and listen to her comment on her battles in real time:

https://t.co/io7CVuDLoN

[DailyPapers @HuggingPapers]

PhysBrain 1.5: Open-source SOTA for Embodied AI

A single 8B model that understands scenes, generates robot actions, and predicts future frames — all as tokens. 72.5 on 28 benchmarks, beating all open models.

[Marc Seal @Kurcyde]

Anyone who says we need to “Slow Down” progress towards AGI is financially incentivized to do so or is uniformed

This is a war against open access to super intelligence. The fear-mongering is manipulation to create support for an unfair competitive advantage against open access

[Marc Seal @Kurcyde]

@0xSero Do you have a recipe you can share for this? I would love to try this on my 4x 6000 Pro Max Q system