Kronk AI

Your personal engine for running open source models locally. Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp, stable diffusion.

[Kronk AI @Kronk_AI]

Quick overview of the #Kronk model server.

[Kronk AI @Kronk_AI]

v1.32.7 Released

Version 1.32.7 brings Malina image generation into the Kronk model server with new API endpoints and BUI screens, a new OpenAI-compatible audio translation endpoint, Wan2.2 S2V audio-conditioned video support in the Malina SDK, a new ukisai/Swift-Qwen3.8-27B-Q8_0 model, and llama.cpp upgrades through b11017.

https://t.co/aAxEHx8TdV

[Kronk AI @Kronk_AI]

Release day!

Expect v1.32.7 before the end of the day.

Lots of great improvements by the llama.cpp and stable-diffusion.cpp teams.

Also full Malina support and more examples.

Remember, you can always run from main to see what's coming and verify your apps and services.

[Ardan Labs @ardanlabs]

What if your Go application could handle AI inference directly?

@goinggodotnet walks through the @Kronk_AI Model Server and how it handles:
→ Local & remote GPU inference
→ Real-time VRAM monitoring
→ In-Memory Caching
→ Hugging Face models
→ Speech + image generation
→ Built-in dev tools & security

🎥 Watch the overview: https://t.co/Fnn4RYyr23

#GoLang #AI #Kronk

[Kronk AI @Kronk_AI]

If you are in the SFO area on Wednesday Octber 7th Bill Kennedy will be speaking at the Go meetup and talking about all things #Kronk and #golang.

https://t.co/96i57CDmOI

[David Hendrickson @TeksEdge]

👀 Wow, kind of aggressive marketing. Send this to your boss.

🎯 AMD is now literally telling businesses they are paying cloud rates for what should be local work.

And they built a calculator to prove it. Link in ALT.

AMD’s new AI Tokenomics Calculator compares ...
☁️ Cloud only
🖥️ Local AI
🔀 Hybrid AI

Example ...
For its “medium” agent workload at 5.74M input + 574K output tokens/user/day, think Claude Code / Codex / Hermes and AMD says ...

500 AMD AI PCs @ 50% local / 50% cloud

Could deliver 40–60% lower 3-year costs than cloud-only.

Go fully local and AMD says savings climb further, with hardware typically breaking even in under 24 months.
But THIS caught my attention 👇

AMD isn’t basing the local side on vague “AI TOPS.”

Its throughput assumptions use ...
🧠 Qwen3.6-35B-A3B
📦 Q4_K_M
📚 262K context / 128K prefilled
⚡ llama.cpp + Vulkan

And it models real hardware, including ...
💻 Ryzen AI 9 HX 470
🧠 Ryzen AI Max+ 395
🎮 Radeon AI PRO R9700

This is where Local AI is headed.

Use frontier cloud models when you actually need them.

Run the enormous volume of drafting, coding iterations, summarization and routine agent work locally where there is no per-token meter running.

Local AI isn't just becoming capable.

AMD thinks the economics may start forcing it onto our desks. 👀

[Jesús Espino @jespinog]

I'm getting my post about the Kronk late by one day (it will be out tomorrow). I was the last week busy at Percona Live, and I have the post almost ready but I don't want to rush it, so I want to give me the opportunity to polish it to the quality that I want.

[ggml @ggml_org]

llama.cpp v0.4.1

https://t.co/Wht1Nm6bDh

[Kronk AI @Kronk_AI]

On the Mac the regular Qwen3.8 27B GGUF was unusable. It would over-think and it was slow.

Well, #ukisai has solved that problem. We are seeing great results using this version of the model.

https://t.co/KCOXNRFwGx

[David Hendrickson @TeksEdge]

Someone fixed Qwen3.8-27B’s biggest problem, overthinking! I gotta try this!!OVERTHINKING.

UkisAI built Swift-Qwen3.8-27B specifically to address this!

The idea comes from Meta research showing that reasoning models can actually reach the correct answer, keep thinking, and talk themselves out of it.

So they identified reasoning-marker tokens associated with those loops and fine-tuned Qwen3.8 to use them less.

Result on GPQA-Diamond 👇

🧠 Qwen3.8-27B
88.38%
median thinking: 6,642 tokens

⚡ Swift-Qwen3.8-27B
88.28%
median thinking: 2,771 tokens

🤯 58.3% fewer median thinking tokens
for just 0.10 percentage points less accuracy. 👈 👀

And check out coding coding preformance ...
💻 LiveCodeBench v6
76.76% → 81.55%
while median completion tokens fell 45.8%.

Terminal-Bench:
66.74% → 65.84%
with 38.7% fewer median tokens

🎯 Swift at xhigh kept GPQA accuracy near normal Qwen3.8 xhigh, while using roughly half the reasoning tokens.

There is already an official GGUF!
📦 Q4_K_M — 18.0GB
🧠 Q5 — 20.2GB
⚡ Q6 — 22.9GB
🚀 MTP included
🦙 runs directly in llama.cpp
💻 LM Studio / Ollama compatible

At 20 tps, every unnecessary 10,000 thinking tokens costs you ~8 minutes. Right!?

⚠️ Caveat: Yes, the benchmarks are creator-reported, and I haven't benchmarked the released GGUFs so can't confirm results.

🔗 HF: ukisai/Swift-Qwen3.8-27B-GGUF

[Quinn Slack @sqs]

Amp is now free to use when you bring your own compute and model subscriptions/keys.

No more limits or fees for BYOK.

https://t.co/8l1jMunBPF

[Kronk AI @Kronk_AI]

Latest versions of Llama.cpp, Whisper.cpp, and Stable-Diffusion.cpp now ready in the main branch. We are keeping this up to date everyday with published tag releases every Thursday.

[David Hendrickson @TeksEdge]

DeepSeek-V4.1-Flash may be showing us where Local AI architecture is heading.

Like Qwen3.8-Flash-Next, it’s another sign that model capacity is becoming increasingly decoupled from active compute.

🎯 It may be a blueprint for how MUCH bigger AI models eventually run locally.

DeepSeek's new V4.1 separated the model into different kinds of memory + compute such as...
🧠 552B backbone
📚 196B Engram memory
⚡ only 8B active/token during input
🚀 only 16B active/token during output

The 196B Engram is especially interesting.

Instead of behaving like normal model weights that all participate in matrix math, it's sparsely accessed through token-based lookup.

For Local AI, architectures like this suggest a memory hierarchy that could look like ...
🎮 accelerator/unified memory → hot compute
💾 host/unified memory → resident weights
💽 SSD → sparse tables / cold parameters

A community 2-bit MLX build on a 256GB M3 Ultra kept ...
🧠 160.9 GiB backbone in RAM
💽 57.2 GiB Engram tables on SSD
...and produced short text generations at around 8.8–9.5 tps.

⚠️ This is VERY experimental and not a production benchmark or stock MLX.

... and DeepSeek attacked another Local AI bottleneck too, KV cache.

V4.1 Flash's global KV cache is only 890 bytes/token.
At 1M tokens, that's roughly 890MB of global KV.

Small active compute.
Huge cheap memory.
SSD-backed knowledge.
Tiny KV cache.

VRAM may be becoming just one tier in the Local AI memory hierarchy. 👀🔥

[Kronk AI @Kronk_AI]

Coming to a release near you. Image generation.

[DeepSeek @deepseek_ai]

🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient.

🔹 Introducing the smallest model in our new architecture family, with native visual understanding.
🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models.

1/6

[dax @thdxr]

it's crazy that a minor update to a flash model is surpassing the previous pro model

deepseek should probably double down on their flash models

it fills a crazy spot in the price/performance spectrum

[Kronk AI @Kronk_AI]

Version 1.32.6 Released

llama: b10896
whisper: v1.9.3
stablediff: master-849-d04e895

Adds runtime library integrity verification endpoints, expands the Malina image-generation SDK with ADetailer refinement, AnimateDiff video generation, and ESRGAN upscaling, and introduces structured download progress for model pulls.

This release also moves CI onto self-hosted GPU-equipped runners, ships a major rework of the speculative-decoding sampler filters, and rolls forward the llama.cpp and stable-diffusion.cpp backends.

[William (Bill) Kennedy @goinggodotnet]

Check out the new #Yzma website and try Yzma in the browser while you are there! #golang

https://t.co/pAkYr0Xh1Z

[Kronk AI @Kronk_AI]

Check out the new #Yzma website and try Yzma in the browser while you are there! #golang

https://t.co/xeHErCyfWI

[Kronk AI @Kronk_AI]

@danicat83 The llama.cpp project moves so fast that this form of versioning doesn't work for Kronk. We essentially just increment the patch number by one until we reach 9 and then increment the minor for every release. I don't expect we will ever increment the major number.