Funball | Kieran

Home lab + self-host enjoyer · inference · virtualisation, networking · hardware enthusiast · building and breaking my own infrastructure. Retired Smite Pro.

[Funball | Kieran @Funball2]

Merlin 3.0.0 is out. My Flash-Next kit is now a proper vLLM fork for RTX 30/40-series & More: ready-made recipes for seven checkpoints, one install, one command.

Flash-Next on 4× RTX 3090, no NVLink, 220 W:
• 184 tok/s decode at the full 262K context
• 6,051 tok/s prefill on a 100K prompt (+12 %)
• 925K-token KV pool, 3 full 262K sessions resident
• Token embedding tables in host RAM
• 8 agents at once: 638 tok/s

[Funball | Kieran @Funball2]

@BottleCapAI Very excited to test this!

[Funball | Kieran @Funball2]

I now have 1 less kidney.

[Funball | Kieran @Funball2]

@boardyai So much untapped potential!

[Funball | Kieran @Funball2]

Merlin 3.0.0 is out. My Flash-Next kit is now a proper vLLM fork for RTX 30/40-series & More: ready-made recipes for seven checkpoints, one install, one command.

Flash-Next on 4× RTX 3090, no NVLink, 220 W:
• 184 tok/s decode at the full 262K context
• 6,051 tok/s prefill on a 100K prompt (+12 %)
• 925K-token KV pool, 3 full 262K sessions resident
• Token embedding tables in host RAM
• 8 agents at once: 638 tok/s

[Funball | Kieran @Funball2]

Measured, not brochure: median of 3 runs, same methodology as the v2 cards, full bench cards in the repo.

Repo: https://t.co/72YU3QXa8N
Models: https://t.co/GHuKXXvYQw

[Funball | Kieran @Funball2]

./setup.sh checks your specs, picks a checkpoint + layout for your GPUs and installs a prebuilt wheel within 5 mins (plus the model download), no compiling. Then merlin serve <checkpoint>.

Profiles for RTX 3090 (with or without P2P), 4090 (24/48 GB), A6000/A40, A100, A30 and CMP 170HX.

[Funball | Kieran @Funball2]

@Karulox @OWCavalry That Artemis skin would of gone hard

[Funball | Kieran @Funball2]

@yarikdevcom @vllm_project @pidotdev 🐐Paving the path

[Funball | Kieran @Funball2]

@thsottiaux What a nice birthday present to wake up to 🐐

[Funball | Kieran @Funball2]

Engine + Merlin checkpoint: https://t.co/Se0VcQLerc · https://t.co/aK4Jk48R2V
Swift 1.5: https://t.co/B0fHhe58Mu

[Funball | Kieran @Funball2]

Flash-Next v2.5.1 is out: Qwen3.8-Flash-Next (125B MoE) on 4× RTX 3090, no NVLink.

• Agent follow-ups resume from cache: 0.96 s to first token
• 8 agents at once: 92.5 % cache reuse
• 925K-token KV pool (+15 %), 3 full 262K sessions resident

Swift 1.5 build updated too 👇

[Funball | Kieran @Funball2]

@iRaffer Congrats bro♥️

[Funball | Kieran @Funball2]

Genuinely surprised at how much traction my Qwen3.8 Flash-Next recipe/kit has gotten before i even tweeted, I put it on github originally to have another location to store proper production candidates. Almost 1.2k downloads on huggingface already 😅

[Coin Bureau @coinbureau]

🇬🇧HUGE: UK government publishes official rules on AI usage as part of its advice on reducing environmental impact -

- Use AI only when necessary, check if a spreadsheet/ search engine can do the job before using AI.

- Choose the smallest model capable of handling the job, such as Gemini Flash instead of Pro or GPT Instant instead of Thinking.

- Prompts should be kept short, with as few as possible used.

- Any AI-assisted work should be personally checked for accuracy.

The guidance also says it should be disclosed when AI helped create or edit content.

[Funball | Kieran @Funball2]

Further improved Running Qwen3.8 Flash-Next:

Updated to vllm 0.30
Opt-in Repeatable greedy output
Still 3 x 262k sessions running at once in the kv pool
Opt-in GPU/NUMA auto-select for other box configs
Same 160-175 tok/s decode

This one is a stability focused release/update.

[Funball | Kieran @Funball2]

Usually tweet once in a blue moon to complain at a company. Deciding to shift gears.

Finally managed to get Qwen3.8-Flash-Next running on 4x RTX 3090s:

807K KV pool, three full 262,144-token sessions at once
162-176 tok/s single stream
Prefill flat 10K to 261K
No NVLink, 220 W

[Funball | Kieran @Funball2]

Checkpoint: https://t.co/aK4Jk48R2V