Espen JD

Cyber Network Engineer | Codex enthusiast | Local AI | RTX Pro 6000 enjoyer

[Espen JD @Snixtp]

I've uploaded Q4, Q8 and BF16 MTP draft models in their own repo

It should work natively with yesterday's llama.cpp merge

https://t.co/Rmbz0b0EVq

[Espen JD @Snixtp]

Q6 MTP is also up for those who want a good balance between size and quality

https://t.co/Rmbz0b0EVq

[Espen JD @Snixtp]

I've uploaded Q4, Q8 and BF16 MTP draft models in their own repo

It should work natively with yesterday's llama.cpp merge

https://t.co/Rmbz0b0EVq

[Espen JD @Snixtp]

@RedBedDread @DJLougen likely because better precision is the safer quality/acceptance default

[Espen JD @Snixtp]

I've uploaded Q4, Q8 and BF16 MTP draft models in their own repo

It should work natively with yesterday's llama.cpp merge

https://t.co/Rmbz0b0EVq

[Espen JD @Snixtp]

New Gemma 4 model

12B, which means it can run on 16GB cards

[Google Gemma @googlegemma]

Meet Gemma 4 12B!

A unified, encoder-free multimodal model designed to bring high-performance intelligence directly to your laptop, and released under an Apache 2.0 license.

Bridging the gap between edge efficiency and advanced reasoning. Here is what’s new with Gemma 4 12B: 👇

[Espen JD @Snixtp]

@TechMDAI Soon I hope lol

[Espen JD @Snixtp]

Mmyes, but misleading.

You’re not using PCIe as VRAM replacement every token. With model splitting, each 3090 mostly reads its own local VRAM, and PCIe is mostly moving activations/tensors between GPUs, its not streaming the whole model every token.

That’s why you don’t see much of a performance difference between x8 vs x16, even tho it’s half the bandwidth.

[Espen JD @Snixtp]

Another 3090 ✅

184GB VRAM

[Espen JD @Snixtp]

@Yeghro Threadripper 7960x and Gigabyte TRX50 Aero D

[Espen JD @Snixtp]

Full model Q4_K_M Step 3.7 Flash: https://t.co/I2nRB7YdOK

[Espen JD @Snixtp]

@thsottiaux do not sunset the Codex name

[Espen JD @Snixtp]

@uzairakrum Real Gs know 5.2 is the goat

[Espen JD @Snixtp]

@TheAhmadOsman Please elaborate on fp8 kv cache on ampere

[Espen JD @Snixtp]

@Liam06972452 @jasondeanlee API only sadly

[Espen JD @Snixtp]

@bsilone I’ve gotten DeepSeek V4 Flash to run with 128K context. ~165GB model alone. Struggling with speed tho because of ds4 engine pop

[Espen JD @Snixtp]

@needmorevram :D

[Espen JD @Snixtp]

I've uploaded my Q4_K_M Step 3.7 Flash GGUF MTP quant on Huggingface

Let me know how it is please

https://t.co/Y5xpvQa70C

[Espen JD @Snixtp]

@taleosmith I can definitely post more :)

[Espen JD @Snixtp]

@faultbugs 3x 3090
Pro 6000
4080s

I’m trying to get q4 DeepSeek v4 flash to run

[Espen JD @Snixtp]

@JakeSamlich llms

[Espen JD @Snixtp]

@mr_r0b0t thank you sir

[Espen JD @Snixtp]

@alpinesnow23 230V EU

[Espen JD @Snixtp]

@alpinesnow23 1600W and 750W

I’m replacing the 750W with a 1000W I got laying around, I’ve just been lazy and not done it yet lol