johmbergsch

small language model

[Sudo su @sudoingX]

a week with the dgx spark, here is what is on it and what i have measured so far. nobody is really talking about this machine and it is quietly becoming the workhorse of my whole stack.

hardware: nvidia gb10 sm_121, 124 gb unified lpddr5x at 273 gb/s, cuda 13.0

models on disk (305 gb total, 9 ggufs):
> qwen 3.6 27b q4_k_m / q5_k_m / q8_0 / ud-q4_k_xl
> nemotron 3 omni 30b-a3b q4_k_m / q8_0 / ud-q6_k / ud-q6_k_xl
> deepseek v4-flash 158b q4_k_m (112 gb, flagship 128gb-tier test)

terminal + shell environment:
> zsh + oh-my-zsh + powerlevel10k theme
> modern cli stack: bat, eza, ripgrep, fd, git-delta, tldr, neovim, fzf, autojump
> 6 tmux sessions actively running for parallel agent work

ml + agent stack:
> llama.cpp built sm_121 against cuda 13
> uv + venv ml stack with pytorch 2.11.0+cu130 (aarch64) + transformers + diffusers + accelerate
> hermes agent v0.11 with codex auth bridge
> opencode for free-model overnight research
> telegram gateway routing to nemotron q8 right now

speeds verified so far:
- nemotron 30b-a3b q8: 56 tok/s gen, 1,300 tok/s prefill, 96% gpu, 33gb in unified
- qwen 27b dense q4: 40 tok/s consistent

90+ gb of unified memory still free. deepseek v4-flash 158b loading next as the real flagship test, multimodal omni testing once mmproj pulls, comfyui install in flight for the diffusion lane.

honestly curious what the actual limit is on this box, i have not hit it yet.

[johmbergsch @le_heifer]

@littmath sounds like opus-4.7

[johmbergsch @le_heifer]

@vladtenev Babel or Alexandria?

[johmbergsch @le_heifer]

the trade off here is clear. Increase i/o efficeny at the cost of having a minimum spec for the hardware. so agree on the storage point.

client diversity and hardware agnosticism is a very different subject for chains that are not eth.

sstore super-linear blowup is a different convo tho whether you want to knee cap devex with aligning costs here

[johmbergsch @le_heifer]

@grugcapital It is what it moo

[johmbergsch @le_heifer]

Hmm

[johmbergsch @le_heifer]

Iiwii

[johmbergsch @le_heifer]

@fede_intern whats wrong with blake3?

[johmbergsch @le_heifer]

so we transitioning from wordcel ai to shape rotator ai soon

[johmbergsch @le_heifer]

@alz_zyd_ Its symbolically dehydrated for succinctness. Learning to rehydrate was a mechanism for understanding.

[johmbergsch @le_heifer]

lean is the future assembly language of reasoning

[johmbergsch @le_heifer]

@hrkrshnn Im glad you studied manifolds

[johmbergsch @le_heifer]

every blockchain idea i think is original, i invariably find a 5 year aged blog post from vitalik detailing the idea.

[johmbergsch @le_heifer]

@ilex_ulmus wonder if ai companies have internal dark forest solutions

[johmbergsch @le_heifer]

@evelovesolive If the computers dont trust each other def not american english. If the computers do trust each other need not be english.

[johmbergsch @le_heifer]

@Sobix1313 @itsmaximilian @meleemarkets @NickPreszler @kaitrias @zen_llama its alright. i cant read (context)

[johmbergsch @le_heifer]

@_jhunsaker @LeverUp_xyz thoughts and prayers

[johmbergsch @le_heifer]

@happysubstack lmao. i failed the marsh mellow test with surfing. surfed out an entire week 2-3 waves knowing the next week would be 4-6 waves.