Nico

Playing with Linux/Unix boxes. (Personal Views) FreeBSD lover. GPG ID Key: 49E42FEB Durée de vie des tweets : entre 24h et 7j.

[Nico @nlienard]

Ca va finir avec une couronne d'alu sur la tete ;-) https://t.co/GwEMF0feKP

[zooko🛡🦓🦓🦓 ⓩ @zooko]

How to steal a GPG key from the next room: https://t.co/DI68b3CWPr By Eran Tromer (https://t.co/75ue55geIm) et al.

[Nico @nlienard]

@ostrisai @ideogram_ai is it possible to do training on a 3090 ?

[Nico @nlienard]

@wildmindai does it work with 4B and 9B ?

[Nico @nlienard]

@BeastySilentd @_SaxX_ ok j'ai trouvé: https://t.co/2WGPl1Gp5Y mais y a pas encore la version 27B

[Nico @nlienard]

@KuittinenPetri @bnjmn_marie following @sudoingX tips, i'm using 27B with Opus Reasoning: https://t.co/QBI9ylk5xB (Q4 K_M for my 3090)

[Nico @nlienard]

@bnjmn_marie try with Opus reasonning, it helps to not loop in thinking : https://t.co/QBI9ylk5xB

[Nico @nlienard]

@BeastySilentd @_SaxX_ tu l'as trouvé ou ? je suis sur https://t.co/T2038HA1ZA en Q4 K_M et j'aimerais l'heretic

[Nico @nlienard]

@_SaxX_ yes, t'as testé celle la ? https://t.co/lWb8RS24uL je cherche la version Heretic en Q4 K_M (3090).

[Nico @nlienard]

@stevibe @sudoingX type "ollama list" to see the GGUF you are using

[Nico @nlienard]

@sudoingX 4token/s with 3090 with these settings any idea why ? set LLAMA_DIR=D:\IA\llama.cpp
set MODEL=D:\IA\models-llama\Qwen3.5-27B-Q4_K_M.gguf
set PORT=8000

"%LLAMA_DIR%\llama-server.exe" -m "%MODEL%" -ngl 99 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0

[Nico @nlienard]

@sudoingX why not adding "-np 1" in that case ? no diff ?

[Nico @nlienard]

@bnjmn_marie @wuxi_China Look at @sudoingX post, he is doing a lot of test on qwen3.5 and according to him , it is the good spot

[Sudo su @sudoingX]

first impressions of qwen 3.5 27B dense on a single RTX 3090.

35 tok/s. from 4K all the way to 300K+ context. no speed drop. hermes 4.3 started at 35 and degraded to 15 as context filled. qwen dense holds. MoE held 112 flat. 3x faster but only 3B of 35B active per token. architecture tradeoff.

Q4_K_M on 16.7GB. native context 262K. pushed past training limit to 376K before VRAM ceiling on 24GB. tried q8 KV cache at 262K, speed collapsed to 11 tok/s. q4_0 KV is the sweet spot. flash attention mandatory.

built in reasoning mode. the model thinks step by step before it answers. full chain of thought surviving Q4 quant. 1,799+ token thinking chains with self correction loops. on a single consumer GPU.

gave it one prompt: "build a realtime particle galaxy simulation in one HTML file." 3,340 tokens. 95 seconds. one shot. ran on first load. full reasoning and coding in the video below.

optimal config if you want to skip the hours of testing:
llama-server -ngl 99 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0

this is just the warmup. octopus invaders is next: 10 files, 3,400+ lines, zero steering. the prompt hermes quit at 22%.

already more impressed than expected. full results coming soon.

[Nico @nlienard]

@iMilnb Llama.cpp est bien vénère aussi niveau optimisation. Tu devrais suivre le compte @sudoingX il fait plein de tests sur tous les qwen3.5

[Nico @nlienard]

@bluetouff C est top nan ?

[Nico @nlienard]

@KaelirRises @OpenAI Which GPU ? Which quant ? How many context length ? With llama.cpp ? 🙏

[Nico @nlienard]

@wuxi_China @bnjmn_marie What about q4_0 KV using llama-server -ngl 99 -c 262144 -fa on --cache-type-k q4_0 --cache-type-v q4_0 for 3090 ?

[Nico @nlienard]

@sudoingX At the end , we can run both model (27B dense or 35B A3B MoE) with 262k context length on a single 3090 . MoE is 90t/s and dense is 35t/s but which one have better quality ? ☺️

[Nico @nlienard]

@MaxiBernard_ C est quel model ? Flux Klein 9B ?

[Nico @nlienard]

@waltonoemi @sudoingX just tried again and i'm at 80 now

[Nico @nlienard]

@waltonoemi @sudoingX what is cache version command ? i run it like that