Jim Hallers

Serial entrepreneur since 2009 who designs, builds and operates bars and restaurants for fun and profit. Prior to hospitality, spent 25 years in tech.

[Jim Hallers @JimHallers]

@0x1ee7 @christopherowen @32k @128k It works just fine using the switch.

[Jim Hallers @JimHallers]

Setup @christopherowen's spark-ds41f-1107 on my 4x DGX Spark setup last night. Same DeepSeek weights, entirely different engine:

prefill 2,700 → 8,141 tok/s @32K
first token @128K: 48.2s → 16.1s
16-stream throughput: 141 → 257 tok/s
242 bench requests, zero failures.

Superb work! It feels like I'm driving a new race car this morning.

Note: It did take about four hours to get switched and validated, so even though it's the same weights, there was a lot to do.

[Jim Hallers @JimHallers]

Artist - this is a three or four spark recipe. And I can tell you is for general business work you need to get from two sparks to four. I ran a GLM model when I only had two sparks, and if that work product is what you need, awesome. But it took me getting to four sparks to handle serious business needs - both in terms of speed, volume and most importantly to me, accuracy. DeepSeek tested the best on my particular workloads.

[Jim Hallers @JimHallers]

What the migration actually took
1. Decided on numbers first. Benchmarked the old stack before touching anything: cold prefill ~2,700 tok/s @32K, TTFT 12.1/23.4/48.2 s at 32/64/128K, c1/c4/c16 block 79.5/128.1/140.8. Decode was already at parity, so the case rested entirely on the ~3× prefill gap.

2. The release tag predates the build fix. spark-ds41f-1107 → cc20ed18 fails build prepare with patch head 2756478b…, expected 320021e3… — git am stops mid-series, leaving .git/rebase-apply. Both tags point at the pre-fix commit; the manifest's recorded head is the stale value; the tree is ed1bdb7a either way. Built from main@ 51be7406, clean first try. That's #1020 (same as closed #952).

3. Site map. nodes-switched.local.json: 4 ranks, management IPs, enp1s0f0np0, roce_gid_index: 3, both HCAs with their subnets. build check accepted it without edits.

4. Host prerequisites — the bulk of the time.

64 KiB kernel packages + linux-modules-nvidia-580-open-…-64k; verified CONFIG_ARM64_64K_PAGES=y and that modules are Canonical-signed, so Secure Boot was a non-issue.
Pinned the stock entry as rollback, then one-shot next_entry into 64 KiB — piloted on a fifth box before touching a rank.
/swap-64k.img had 4 KiB headers; mkswap --pagesize 65536 while inactive, then configure-swap + the kernel-memory service, THP never, min_free_kbytes=45166.
Scoped passwordless sudo: sudo -n true, memguard start/stop/reset-failed, systemd-run --unit=spark-ds41f-memguard*, and sudo -n python3 — the doctor's remote kernel inventory execs inline python as root.
The one that cost us a night: the launcher also calls sudo -n systemctl is-active --quiet spark-ds41f-memguard.service. Without that exact rule, a perfectly healthy guard reports "exited before completing its initial protected sample" and the rollback kills it. Worth documenting beside the other three.
gdm3 off on two nodes (display stack eats unified memory), and a Host block for the head's own IP — self-SSH had no identity to offer.
5. Checkpoint layout was the real surprise. The profile mounts the HF hub snapshot; our old stack kept only per-rank shards (2 of 48 files per node, none on the head), so the hub path simply didn't exist. Pulled the revision once, replicated 56 blobs over RoCE at ~500 MB/s, 17 min/node. Also: cluster.json has revision 1bae0e1089… where the repo's is 1bae0e1088… — one digit, and hf rejects it outright.

6. Build costs downtime by construction. build image --apply refuses while a DS4.1 container runs, so unlike a pure staging build you must stop serving first. Launch: API-ready in 117 s warm / 626 s cold.

Result: prefill 2,700 → 8,141 tok/s, TTFT @128K 48.2 → 16.1 s, c16 141 → 257 tok/s, 242 requests zero failures.

Two operational consequences: --max-parallel-prefills 1 serializes cold prefills (c16 TTFT 482 → 512 ms while throughput doubled) — we re-spread every cron onto its own minute for exactly this. And the served name changed to deepseek-v4.1-flash, which breaks any pinned client.

We ran stock UVM, no memory saver — headless boxes, MOK enrollment needs a console. ~20–23 GiB available per node after load.

[Jim Hallers @JimHallers]

@Agantuk01 @mdanesth @RyanJAyala I use NOW brand on Amazon, the 1000mg capsules. There are several others that are likely just as good there to pick from as well. I tried bulk powder NAC once as well, not good tasting.

[Jim Hallers @JimHallers]

Well, I have a strong opinion that America is being poisoned in our food (glyphosate and seed oils) and by people no longer getting enough sunshine - so almost everyone’s mitochondria are damaged to some degree. It’s almost impossible to avoid it.

Just look around. People today in the United States do not look like they did in the eighties. And they ate too much back then as well. What’s the big difference? The issues I listed above all got cranked up on high the last thirty years.

So your body needs all the garbage collection it can get. Hence GlyNAC. Or you can inject glutathione but it’s expensive.

Besides using Function Health these days to track way more blood markers than I used to I also have taken a mitoswab test twice. One part of my Krebs Cycle was operating at 9% on the first test.

The big issue is there is no money to be made pushing NAC and Glycine or NAD+. Heck, we can’t even get the government to recommend sunlight or Vitamin D3…

[Jim Hallers @JimHallers]

@mucki777 @RyanJAyala For NAD+ you either try NMN or NR supplements or get it by IV (too expensive and time consuming) or you inject it. I did not find the patches to be effective.

[Jim Hallers @JimHallers]

@Sad_Liberal @RyanJAyala Now 1000mg - under $20. I’m sure several brands are good. Just get 1000mg and not 600mg.

[Jim Hallers @JimHallers]

@breathe123 @RyanJAyala Now 1000mg

[Jim Hallers @JimHallers]

@keschet @RyanJAyala Yep. I started with patches. And tried NMN and NR products. But for me injections seem to deliver results. Now is it the GlyNAC just getting even better? I do know it’s 100% for real. As for NAD+ I just believe in keeping your “tank” full and the body does good things.

[Jim Hallers @JimHallers]

Years ago when I started there were a bunch of people posting high dose NAC will cause problems. I started with 2000mg and moved to 4000mg and got good results so I’ve not increased it from there. Maybe it would be even more amazing if I went to 7000mg a day. But I have gone up on Glycine with all the 2026 reports of its benefits.

[Jim Hallers @JimHallers]

@jrsygrl4444 @RyanJAyala Search NAC 1000mg on Amazon

[Jim Hallers @JimHallers]

@bear_in_mind1 @RyanJAyala Now 1000mg - price is under $20 for each.

[Jim Hallers @JimHallers]

7/7
Takeaways:
- Never let an LLM add up long columns. Give it a calculator or code tool.
- Test effort levels on your own work. Defaults can mislead.
- A fast wrong total costs more than a slow right one.

One run per prompt, quantized builds on DGX Sparks. Not a lab test.

[Jim Hallers @JimHallers]

Most local-LLM tests are about code. I run restaurants and bars, so I tested DeepSeek V4.1 Flash and GLM-5.3 (Flash and full) on business work: P&Ls, prime cost, schedules, vendor emails, menu engineering, and an 874-line invoice ledger. What surprised me:

[Jim Hallers @JimHallers]

6/7
5. Each caught things the other missed. GLM flagged a "weekly" linen service billed 12 times in a month; DeepSeek never did. DeepSeek on high was the only one to notice August had 5 Sundays and September 4, so raw totals overstate the drop.