Niko Veit

[phone] (WhatsApp)
[email protected]

[Niko Veit @coldniko]

Multi GPU support is now live on Github

[Niko Veit @coldniko]

ROCm + Dual GPU support = Tomorrow

https://t.co/83diJSeb8I

[Niko Veit @coldniko]

Special thanks to @Ribbon462 for renting his PC out 🫡

[Niko Veit @coldniko]

Multi-GPU Support + Tensor Parallelism supported!

Prompt processing gain is around 20%, output gain around 5%-6%. Will be optimized further soon.

ROCm support next 👀

https://t.co/83diJSeb8I

[Niko Veit @coldniko]

@Ipsi_Cust @itsmissglitch abbliteration can be enabled in "Chats" tab under "Experimental speed projection" toggle. It will uncensor any model you run with the engine

[Niko Veit @coldniko]

Who wants to run Qwen3.8 Flash Next with 8GB GPU and 16GB of RAM? 👀

[Niko Veit @coldniko]

@marektutoky Write me!

[Niko Veit @coldniko]

To improve Qwen3.8-Flash-Next engine I need:

- A person with AMD GPU + at least 40GB RAM
- A person with dual GPU setup + at least 32GB RAM

All I need is SSH access for a day on each to pin-test the engine and adapt it for dual gpu’s and amd rocm.

Please share, if you’re not the one! 🫡

[Niko Veit @coldniko]

@itsmissglitch A person with low specs and no funds for AI stuff is happy at 10 t/s, trust me on that one :)

[Niko Veit @coldniko]

@smogporter Amd support in next 24hrs

[index @AgentArchetype]

New one-shot voxel pagoda build — straight from my very first launch with Strata by @coldniko.

Running the IQ3_S GSQ-RCO quant from ISTA-DASLab on a 5080 + 64GB DDR4 and sustaining ~36 tok/s through Hermes.

Absolutely wild performance from hardware that barely even registers as “serious” by homelab standards.

[Niko Veit @coldniko]

@MgkMshrmBrkfst 1x 5090

[Niko Veit @coldniko]

Just doubled the prompt processing speeds from 539 token/s to 1290 tokens/s.

Qwen3.8-Flash-Next on consumer GPU is now blazing fast.

Next on the list is further optimizations on kernels and swapping which should boost results by 3X-4X.

https://t.co/83diJSdDja

[Niko Veit @coldniko]

@0xSero Intel goes brrrrr

[Niko Veit @coldniko]

@unrivaled_rr1 @kiliczsh For now its between 20-30 tps, prompt processing gets hit at 380 ceiling

[Niko Veit @coldniko]

@FL511_ There is :))

[Niko Veit @coldniko]

@MgkMshrmBrkfst All models are uncensored - click on "Chat" tab and turn on "Experimental Speed Projection"

[Niko Veit @coldniko]

@MarketW1zardry Yes

[Niko Veit @coldniko]

@badguyty Tomorrow. Dont worry

[Niko Veit @coldniko]

@nawariokra IQ3_S or Coder version are goated!

[Niko Veit @coldniko]

@megakilo https://t.co/lM7QmvdsgV

Here you go!

[Niko Veit @coldniko]

@megakilo Yes

[Niko Veit @coldniko]

@JZGibbons @Luisezpeleta You gotta do what you gotta do

[Niko Veit @coldniko]

Run Qwen3.8-Flash-Next-GSQ-RCO-Coder on 32GB of RAM (64GB RAM+VRAM needed for full 265K context)

44 tps output / 1,300 ppts - Enjoy coding fast on consumer hardware!

https://t.co/Aki3ZOCDKX

https://t.co/83diJSeb8I

[Niko Veit @coldniko]

@ScottyT72 @zackdell_dev 👀