Matty Horne

LUFC. Golf. Skiing. Building @FestivalOfSnow. IT Manager. Trying to build an AI Agency #BrumaliaStudios #BuildInPublic

[Gandalf Stormdrain @auroter]

Looks like Anthropic rug-pulled us again. Completely on schedule.

Several days into the subscription-based trial of Fable, they flipped a switch. They switched quants, they changed up their KV cache to INT-0, they turned whatever degenerate knobs they have in their infrastructure, and it's dumb again.

They do this with every single model release. I'm not sure if you guys have noticed. They drop a new model, it's smart for maybe a week, and then something changes and it becomes incredibly stupid and incredibly difficult to work with.

The entire world is learning what happens when you don't control the weights. You get rug pulled every single time.

This is why closed source AI will lose.

Closed-source AI companies are incentivized to be evil. They're incentivized to be predatory. They're incentivized to experiment on their users like lab rats...

Because they know the writing's on the wall. That's why they're stealing everyone's IP. Look at all the "products" they've launched. They did this with our data. The only way for them to survive is by playing dirty.

I hope that from these last few days with Fable, everyone has enjoyed a preview of what awaits us in the open source utopia that is the not-so-distant future.

GLM5.2 was trading blows evenly with Fable, the first couple days we had access to Fable. Even at a ridiculously low quantization level of 3.8 bits. Now, it's better than Fable.

They make it dumb so that you'll give them your data. They make it dumb so that you can train the next model, by telling it what it's doing wrong. You are helping them oppress you. You are helping them scam you. You are helping them steal everyone's intellectual property.

This is why open source will win. We are only one generation of open source models away from them taking the lead.

You take out the alignment garbage, it gets smarter, smaller, faster.

Closed source will lose, open source will win, and it's already in the bag.

[Matty Horne @mattyh575]

@FabrizioRomano Boring

[Matty Horne @mattyh575]

@sudoingX 8GB bring it on. I want to develop @festivalofsnow locally. Ha ha

[Alok @analogalok]

you're paying $20/mo for something your $500 GPU can already do.

Gemma 4 26B A4B QAT MoE + Hermes Agent running on a single RTX 4060 (8GB VRAM).

Built a vision capable, 100% free, 100% local, private AI assistant that lives in my Chrome browser. No API keys. No cloud. No subscriptions. 100% vibe coded. 0% handholding.

It has full context of whatever's on my screen can answer questions, summarize pages, extract data, and see images. Same local model handles everything, no external calls, ever.

keep reading for the model and hermes agent tips i learnt while building this locally.

Here's the exact setup for anyone running local LLMs on 6-8 GB VRAM:

llama.cpp server flags (on my NVIDIA RTX 4060 8gb VRAM):

-m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --cache-type-k q8_0 --cache-type-v q8_0 -c 150000 --port 8080

Throughput with quantization:
Prefill: 200-250 tokens/sec
Decode: 20-25 tokens/sec

reduce context if oom on 6 gb vram card.

Key learnings:

- Quantize KV cache to q8 for faster prefill/decode. Prefill goes from 100-150 (unquantized) to 200-250 tok/s (q8).
- But watch out, once actual context grows past ~50k tokens on high entropy workloads, q8 KV quantization can cause hallucinations. Low entropy workloads are mostly unaffected. If you see it happening, drop the quantization. This is common across all local models.
- In Hermes Agent settings -> Memory & Context, bump compression threshold from default 0.5 to 0.7. Default triggers way too frequent context compression and eats time.

Up next: add persistent memory, web search, tool calling, streaming output and whatever you suggest.

Running a 26B MoE with vision + 150k context window on 8GB VRAM would've sounded impossible 6 months ago.

Works the same on the NVIDIA RTX 3060 Ti, 3070, 4060 Ti, 5060, 2080, or any 8GB card. VRAM is the only requirement.

Local AI agents are closer than people think. You just need to know where the knobs are.

Model's Unsloth quant hugging face link in the comments.

Have you tried Hermes agent by @NousResearch yet?

What are you building with local LLMs? Drop it below, let's see what this community is shipping.

[Matty Horne @mattyh575]

Trying to create a client oauth ID setup in GCP and running into errors. AsK @GeminiApp to help....2 days later, still no help. All the screens that Gemini are telling me to go to do not exist anymore.

[Matty Horne @mattyh575]

@boxmining @NousResearch Hermes Agent on a VPS with @garrytan gBrain and @mattpocockuk Skills building @festivalofsnow huge learning curve for an old developer used to Powerbuilder and VB Code 😆

[Matty Horne @mattyh575]

Marvelous

[Hamster of Orion @hamsteroforion]

Landscape Generator 2.0 is live now with a full day & night cycle.

I tried to give each time of day its own mood. Height fog still needs some love though...

New stuff:
Stars and moon in the night sky.
Night rim light added.
Distance fog added.
Height fog added.
Sun glare effect.
Better mobile version UI.
Godrays converge from the sun position.
Fixed a GTAO bug where foliage vertex animation wasn't accounted for.

Try it 👉 https://t.co/0gbtl2dnH1

#indiedev #threejs

[Matty Horne @mattyh575]

@petergyang VPS everytime. At some point I would put an agent like Hermes from @NousResearch at root to look after all my infra also. Currently it runs in a sandboxed docker container with me in the middle.

[Matty Horne @mattyh575]

@sudoingX @festivalofsnow

[Matty Horne @mattyh575]

@iamlukethedev I use subscription models for Claude, Codex, Minimax, @NousResearch and GLM. I have tried to get my Hermes agent to track usage on their websites to no avail. I would love to know if anyone has managed it and the agent could tehter when to use such models that are low on usage.

[Teknium 🪽 @Teknium]

@NousResearch We are working on benching various combos of open source models to see if we can get Opus levels with much cheaper models as well, stay tuned!

[Festival of Snow @festivalofsnow]

Every range. Every peak. One planetary canvas for the global mountain authority. Spin the globe. Lock your sector. The mountain, decoded.

[Festival of Snow @festivalofsnow]

THE DUAL SOUL.
Two lenses. One mountain.
SOUL: an expensive travel journal — glacier white, serif, quiet.
SHRED: a tactical ops room — carbon black, live intel.
Same world. Different operator.
The toggle transforms everything.

[Festival of Snow @festivalofsnow]

The mountain, decoded.

Festival of Snow is a ski resort explorer. Every rider sees the mountain differently.

SOUL: heritage, long runs, wide vistas.
SHRED: carving, terrain park, deep powder.

Same mountain. Two ways in.

Coming soon.

[Matty Horne @mattyh575]

Loving Hermes. The self learning loop is amazing, linked with @mattypollock skills the development cycle is sound so far.

[Nous Research @NousResearch]

Hermes Agent v0.17.0 - The Reach Release

Changelog below:

[Nous Research @NousResearch]

Hermes Agent v0.17.0 - The Reach Release

Changelog below:

[Nous Research @NousResearch]

Hermes Agent is so good right now…

[Teknium 🪽 @Teknium]

Hermes v0.17.0 is now available

The Reach Release

[Matty Horne @mattyh575]

@HermesAgentTips Jeez you guys are missing out. Always after the new models. Come on does anyone review what AI produces? The frontier models -1 can always do what you want.

[Matt Pocock @mattpocockuk]

Announcing mattpocock/skills v1

- Achieved a 63% reduction in token cost for skill descriptions
- Split skills into model-invocable and user-invocable skills, adding /codebase-design, /domain-modeling, and /grilling
- (UPDATED) /writing-great-skills - rewritten from the ground up, encoding my skill-writing best practices
- (UPDATED) /diagnose -> /diagnosing-bugs - now model-invocable, awesome for fixing hard bugs
- (NEW) /ask-matt: a router skill that teaches you how all the engineering skills work together