phon xay

Digital artist - specialize in generative art and soon artificial intelligence in art and music

[shmidt @shmidtqq]

https://t.co/oTElGPPX96

[Alok @analogalok]

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec

If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware.

Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card.

The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?"

Today, I’m delivering exactly that.

I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!.

If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed.

The performance metrics are astonishing:

- 20 tokens/sec flat decode throughput.
- Stable, flat decode speed even with massive prompts.
- I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame.

# What about prefill?

Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable.

And this is running completely without Multi Token Prediction (MTP) active.

How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4.

The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse.

# The Test Setup:

CPU: Intel Core i7

RAM: 16GB System RAM

GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM)

# The Secret Sauce (The -cmoe Flag)

To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp.

This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache.

It prevents VRAM spillage and holds the throughput rock solid.

# The flags:

-m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v

Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking.

Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

[Pi Update @CoreNews_2]

Help bring more AI-created apps to the Pi ecosystem and enter a raffle to win Pi Network merch!

Pioneers are invited to introduce Pi App Studio to vibe coders and AI-builder communities around the world.

Why?
Because as AI makes building apps easier, competition for human attention and user acquisition becomes more intense.

Pi App Studio helps creators connect AI-built apps to:

60M+ engaged Pioneers
Built-in payments
Identity infrastructure
Pi Ad Network
And more!

Find a relevant builder community, share how Pi can help creators reach real users, then submit your post through the Pi App to enter the raffle.

By helping more creators discover Pi, Pioneers can help expand the range of apps, tools, and services available in the Pi ecosystem. A stronger app ecosystem gives Pioneers more ways to use Pi, support useful products, and participate in real utility.

Go to the Pi mining app to learn more.

[phon xay @ddragon]

@CoreNews_2 A good get start on pi for developer would be nice 👍

[Akshay 🚀 @akshay_pachaar]

Google just dropped a new LLM!

You can run it locally on just 8GB RAM.

Let's fine-tune this on our own data (100% locally):

[phon xay @ddragon]

@akshay_pachaar Why are we torturing each other by not putting useful information on the blog- for god sake stop using x, y, or z social app!😆😆😆

[phon xay @ddragon]

It's really piss me off to see “verify you are a human “ delay on website these days, I think this got to STOP! #verifyyouareahuman

[phon xay @ddragon]

@heyshrutimishra MIT Edd kick you off the course after the free time expires unlike Coursera which allows you to learn via audit mode - better

[Massimo @Rainmaker1973]

How a book written in 1910 could teach you calculus better than several books of today

[Calculus Made Easy, by Silvanus P. Thompson, 1910 - full text pdf: https://t.co/W4gspJoGQc or with the table of contents: https://t.co/55dkZAeGmv]

[クリ Cristian Peñas @kuri_pl]

This is a completely 2D image generated by #Midjourney, and now I have full control over the 3D transition using some magic in #Unity 🪄

[ChrisHandzlik @chris_handzlik]

Fast Script Reload - Hot Reload implementation for Unity is now open source!

I was really chuffed to see FSR selling over 1000 copies on Asset Store in February and going to #1 best selling.

Hope this way even more game devs can get it:
https://t.co/yE7DhusTEl

[Milan Jovanović @mjovanovictech]

How do you use 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 EF Core 𝗗𝗯𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘀 in a single application?

Here are a few scenarios of why you would want to create multiple DbContexts.
- Multiple databases
- Separating concerns
- Modular monolith

#dotnet

[phon xay @ddragon]

I am sending you 1π! Pi is a new digital currency developed by Stanford PhDs, with over 23 million members worldwide. To claim your Pi, follow this link https://t.co/b8bnYEnfTf and use my username (phonx88) as your invitation code.

#crtpto, #PiNetwork

[hackaday @hackaday]

Commodore 64 Emulator in VR Delivers a Full 80s Experience https://t.co/r1mz85mbXa

[phon xay @ddragon]

@kenshirriff I once at a computer auction and the vax machine was auction start $1 but no one wants it and I was so so sad that I don't have space to put it -Arhhhh

[CoinDesk @CoinDesk]

Anyone can download an exact replica image of @beeple’s artwork by googling its title. What is comparatively more difficult to do is fool people into thinking you own the original art creation.

That’s what “Monsieur Personne” is trying to prove.

https://t.co/vpqw5oCzPa

[phon xay @ddragon]

you can now buy cryptocurrency in Australia and New Zealand here! https://t.co/mdoi9DbFGm

[phon xay @ddragon]

The Linux Foundation Announces Hosting of AsyncAPI https://t.co/2MqbELWIxY via @InfoQ

[phon xay @ddragon]

Get my art printed on awesome products. Support me at Redbubble #RBandME: https://t.co/h2AmuWvhdH #findyourthing #redbubble

[phon xay @ddragon]

Get my art printed on awesome products. Support me at Redbubble #RBandME: https://t.co/SnOcxpgxob #findyourthing #redbubble