Wai Park Soon 🍥🕙

[Wai Park Soon 🍥🕙 @pswai]

@Sink1737182 Thanks again for the invite!

StartLux made it in, and your answer on the training split is linked so readers get the full picture. Happy to rerun when you release a new version.

https://t.co/ZWTYfBLeWD

[Wai Park Soon 🍥🕙 @pswai]

1/ I tested Jev against a free local model on one narrow task: checking 300 synthetic statement lines against their extracted records.

Accuracy:
• Ten lines of code: 0.99
• Jev, tested through its API: 0.89
• Laya, local on my Mac: 0.59

[Wai Park Soon 🍥🕙 @pswai]

3/ Also tested: StartLux (thanks @Sink1737182), Cloudflare's new Clef, Decider and Laya.

Write-up, chart and caveats:
https://t.co/ZWTYfBLeWD

[Wai Park Soon 🍥🕙 @pswai]

1/ I tested 14 open models against Jev on my 64GB Mac, on one narrow task: catching bad extractions in 300 synthetic statement lines.

Two kept pace: my everyday local model, Qwen3.8 Flash-Next, and Kev 9B.

Neither could be told apart from Jev on this sample.

[Wai Park Soon 🍥🕙 @pswai]

2/ The surprise: Flash-Next has no decision training. I just read the probability of its first answer token.

Kev 9B is the practical pick: about 17 GiB while running, against about 44 for Flash-Next.

[Wai Park Soon 🍥🕙 @pswai]

@Sink1737182 Thanks, that's clear. I'll note it in the write-up and send it over when it's out.

[Wai Park Soon 🍥🕙 @pswai]

@Sink1737182 Ran StartLux on my M4 Max: your typed-decisions numbers reproduced closely (0.758 / 0.798 / 0.804).

One question before I publish: scores above that set's 0.735 teacher ceiling usually mean learning its labels. Was any typed-decisions data in your training mix?

[Wai Park Soon 🍥🕙 @pswai]

3/ Write-up, data and caveats:

https://t.co/hJuvdJ7u4j

[Wai Park Soon 🍥🕙 @pswai]

2/ The script wins because it knows my data's exact layout. Unfair, and that's the point: if you know the structure, code beats a model.

Jev told good lines from bad. Laya was ~17× faster per call than Jev from Singapore, but barely separated them, outside its training.

[Wai Park Soon 🍥🕙 @pswai]

4/ Write-up: https://t.co/XluiOvz94h

Raw logs + reproduce script: https://t.co/piILuDAevK

Upstream PR: https://t.co/VAvR4w6b6y

@antirez: 64GB numbers for the official page, if useful.

[Wai Park Soon 🍥🕙 @pswai]

1/ A 125B MoE model at ~41 tok/s on a stock 64GB Mac.

- DwarfStar's official ds4-bench
- Full 2K→64K sweep
- M4 Max 64GB
- MacOS default settings
- --prefill-chunk 2048

~580–620 t/s prefill, ~41 t/s gen, nearly flat to 64K. 3 of 3 runs.

Numbers + raw logs ↓

[Wai Park Soon 🍥🕙 @pswai]

3/ The number I cared most about on 64GB: cold start to first token.

Qwen: 23.7 s cold → 3.7 s warm.

That's 41.7 GiB of weights coming off the SSD, paid once per server start.

[Wai Park Soon 🍥🕙 @pswai]

I have moved to DwarfStar! Huge thanks to @antirez and @ivanfioravanti !

Pleasant daily usage with Hermes on my M4 Max 64GB. Recipe, journey, some catches.

https://t.co/3Du24L0ETx

[Wai Park Soon 🍥🕙 @pswai]

@ommuruganthunai @ivanfioravanti @_p0ly_ Mostly yes.

--think and --nothink work for Qwen3.8 on the CLI (on by default = the card's xhigh). --think-max = --think here; --think-level is DeepSeek-only.

[Wai Park Soon 🍥🕙 @pswai]

@ivanfioravanti @antirez Overnight on the M4 Max 64GB: prefill up two thirds (534 vs 322 t/s), decode about the same, starts in 21 s, same memory footprint.

Good enough that my daily Hermes runs on it since this morning.

One restart-cache finding filed as #1053.

Write-up soon.

[Wai Park Soon 🍥🕙 @pswai]

@ivanfioravanti @antirez Great, let me try out!

[Wai Park Soon 🍥🕙 @pswai]

@alexndrxc @MysteryHacker1 @ivanfioravanti @antirez All of it is AGENTS.md. That repo's AGENTS.md is 96 KB, and the context tier went 0 → 98 KB. Skills index was 9 KB in both runs (always-on, cwd doesn't change it); coding guidance added ~1.5 KB. Hermes reads AGENTS.md, CLAUDE.md, .cursorrules and .hermes.md from the cwd.

[Wai Park Soon 🍥🕙 @pswai]

My daily assistant is now a 2-bit 125B model on a 64GB M4 Max: @ivanfioravanti's Qwen3.8-Flash-Next quant on @antirez's DwarfStar. Settings, costs, 2 bugs.

https://t.co/ikLj9Gbj0Y

[Wai Park Soon 🍥🕙 @pswai]

@MysteryHacker1 @ivanfioravanti @antirez Most efficient way to tune the config is to use Claude/Codex to actually benchmark your machine. Then ask them to monitor your actual usage to adjust to your use case.

[Wai Park Soon 🍥🕙 @pswai]

@MysteryHacker1 @ivanfioravanti @antirez Check your cwd. Start it from a clean directory.

Hermes folds AGENTS.md/CLAUDE.md and similar files from the working directory into the prompt. Mine jumped from 29 KB to 128 KB when started inside a repo.