Nathan Wilson 🌋🏝️

#CyberSec #buidl #secp256k1 ⚡⚡

[Nathan Wilson 🌋🏝️ @NathanW1014]

Permissioned vs Permissionless explained in 2 photo's

[Nathan Wilson 🌋🏝️ @NathanW1014]

https://t.co/shD8VhsjRU

Small update:
-fix segfault in multi-slot through llama-server
-build fix, for anyone build compile issie when 'DGGML_RPC=ON' is set

[Nathan Wilson 🌋🏝️ @NathanW1014]

https://t.co/CBtu0WFjT4
v0.7.6 is out, built in parallel with halo-box (PR coming)

- Prompt processing +16% to +31% on standard models tested
- Flash-Next PP flat with depth now, +34% at 32k
- MTP speculative decoding ~9% faster on 27B with slightly better acceptance

[Nathan Wilson 🌋🏝️ @NathanW1014]

@cozybearlog Ive taken on the recommendations in the latest release

https://t.co/yQBvdAgtLE

[Nathan Wilson 🌋🏝️ @NathanW1014]

https://t.co/CBtu0WFjT4
v0.7.6 is out, built in parallel with halo-box (PR coming)

- Prompt processing +16% to +31% on standard models tested
- Flash-Next PP flat with depth now, +34% at 32k
- MTP speculative decoding ~9% faster on 27B with slightly better acceptance

[Nathan Wilson 🌋🏝️ @NathanW1014]

New toolbox release: https://t.co/oxTkZWkwjF

- Qwen3.8 Flash-Next: faster PP and TG at long context (+35% TG, +59% PP at 32k)
- DSv4 with DSpark: +17% PP with the drafter attached
- Unsloth shared MTP heads for Flash-Next now load

[Nathan Wilson 🌋🏝️ @NathanW1014]

https://t.co/CBtu0WFjT4
v0.7.6 is out, built in parallel with halo-box (PR coming)

- Prompt processing +16% to +31% on standard models tested
- Flash-Next PP flat with depth now, +34% at 32k
- MTP speculative decoding ~9% faster on 27B with slightly better acceptance

[Nathan Wilson 🌋🏝️ @NathanW1014]

@cozybearlog It would be nice, but its not feasable on each release. The only things changed per test sweep was the engine version. I do agree the launch parameter, driver, model, IOMMU, power profile should be mentioned in the notes, and those settings can then be infered/transposed.

[Nathan Wilson 🌋🏝️ @NathanW1014]

@cozybearlog Thanks,
Qwen4 engine is brand new and has a ton of work to improve the new mechenisms it introduces.

The percent gains are attributed to each patch in the release notes, the patchs may be more or less sensitive to power setting/iommu but theyre not the gain seen between release

[Nathan Wilson 🌋🏝️ @NathanW1014]

@xaler2 @ d32k
V7.4.1 : pp 182, tg 19.6
V7.5 : pp 211, tg 25.0
Upstream: pp 66, tg 17

@ d64k
V7.4.1 : pp 66, tg 12.5
V7.5 : pp 100, tg 19.8
Upstream: pp 38, tg 12

[Nathan Wilson 🌋🏝️ @NathanW1014]

@xaler2 I only have a 64GB box, so numbers below are relative: lower ub (-ub 512), and offloading some MoE layers to CPU (-ncmoe 8), on a 128GB box that can run higher ub (-1024, or -2048) and wont need ncmoe all numbers will be higher.

[Nathan Wilson 🌋🏝️ @NathanW1014]

@FrameworkPuter @dcapitella I did all this on a Framework Desktop

Thanks @dcapitella for the mention in the artical and thanks @FrameworkPuter for building solid hardware !

[Soju @sojufx]

Local AI addiction starts innocently. I just want privacy.
Then you’re downloading 200GB checkpoints at 2am, comparing 4-bit quants, calculating memory bandwidth over breakfast, and calling a new GPU infrastructure.

Anyway, one last benchmark and I’m going to bed ☺️.

[Flex @flexmeow]

Flex 101: E2

demystifying the most unstoppable loop in the world, which also happens to be paying >120% APY

@LiquityProtocol's yBOLD

🐈🧵⏲️!

[Nathan Wilson 🌋🏝️ @NathanW1014]

@jauneclaude @dcapitella @rc_morano @Italianclownz @ciruai Thanks for the mention!

[Nathan Wilson 🌋🏝️ @NathanW1014]

@Gabriel12061312 @AIatAMD @FrameworkPuter Ive found 1 specific improvement for ROCm 'HIP tile-dequant KV' that increases TG at depth (Q8_0) (+128% @ 32k / +232% @ 64k),

Im working with another dev whos pushing out a ton of improvements on ROCm,

I found ROCm unstable, but ROCm 10 is interesting

https://t.co/XPc4bZwajl

[Nathan Wilson 🌋🏝️ @NathanW1014]

Fastest way to run Qwen 3.8 Flash on an AMD Strix Halo

https://t.co/9sGU4YQfXm

@AIatAMD @FrameworkPuter

[Ahmad @TheAhmadOsman]

I need Opensource AI releases to slow down not because of safety reasons but because my compute is busy and I cannot keep up with all the releases

What an amazing timeline we ended up in for Opensource AI

[Nathan Wilson 🌋🏝️ @NathanW1014]

@Gabriel12061312 @AIatAMD @FrameworkPuter 27703

[Nathan Wilson 🌋🏝️ @NathanW1014]

@FrameworkPuter @AIatAMD Note, the bigges improvements are at depth, (where it counts!)

[Nathan Wilson 🌋🏝️ @NathanW1014]

Ive sweeped almost all localmaxxing for Strix Halo runs (on an older build!)
https://t.co/S1o0seWDho

Id be keen for someone to submit a DSV4 run (ive built this all with a 64GB framework desktop, so I cant submit the best perfoming run)

@FrameworkPuter @AIatAMD

[Nathan Wilson 🌋🏝️ @NathanW1014]

Fastest way to run Qwen 3.8 Flash on an AMD Strix Halo

https://t.co/9sGU4YQfXm

@AIatAMD @FrameworkPuter

[Nathan Wilson 🌋🏝️ @NathanW1014]

@elderorb @AIatAMD @FrameworkPuter Yeah it does, i dont build a Windows executable, but its avaialble if you eithe rbuild it from source, run it through docker, or use the community build: https://t.co/QYf5Y1ETDV

[Nathan Wilson 🌋🏝️ @NathanW1014]

@Gabriel12061312 @AIatAMD @FrameworkPuter The whole repo is based on improving llama.cpp, have a look at the table, and check with some other models (dsv4 works extenely well/fast),

Ive upstreamed a pp gain for Q8, with other supported wuants to come. I have an upstream PR inflight now that improves f16 by 2x at depth

[Nathan Wilson 🌋🏝️ @NathanW1014]

@0xzz3 @AIatAMD @FrameworkPuter Use Vulkan