CocaKova

I run models at home and build the boring tools that make that usable.

Learn ➡️ Do ➡️ Advance ➡️ Repeat 🔄

[CocaKova @Jonnykov]

Big Keryx update on my phone tonight. 2.10.

- The sidebar finally makes sense. Sessions on shelves by day, and the folds stick.

- Filters for what needs me, what's running, what I haven't read.

- Project sessions stay in their project. A "show older" row gets me back to chats from weeks ago.

- One search box for everything: sessions, commands, settings.

- Settings are flat and searchable. Type "vibrate", it takes you to the row.

- When a turn fails it tells me what broke and offers a retry.

- Tap the context ring to see what's eating the window.

- Long-press the last reply to take the whole exchange back.

- New sessions open on the model I last picked.

- Chats open instantly. Like, before the gateway has even answered.

All of it against a Hermes gateway on hardware I own. That part doesn't change.

[CocaKova @Jonnykov]

@iAjittiwari Tell me about it 😅 the PR absorbtion was going too slow. Which I don't blame them - got a lot on the plate, but yeah it was a little bit tiresome haha

[CocaKova @Jonnykov]

@loktar00 Just a display unit? Back in the day that calls for a discount! 😂

[CocaKova @Jonnykov]

@TheAhmadOsman Bro says local weights are trash then turns around and releases one

[CocaKova @Jonnykov]

Qwen3.8-27B NVFP4 on the Garden Trials, one Spark:

• Orbit 5/5 at low effort, two perfect JSON parsers at medium

• TOML 0/10. Seven runs reasoned 109k+ chars of clean planning, then stopped and wrote nothing

• Best TOML parser still hit 0.8884 vs the last climber's 0.858 (interestingly)

Full report: https://t.co/2vljdha7WM

[CocaKova @Jonnykov]

@TheAhmadOsman @BrandonMusicKy @loktar00 @mr_r0b0t @usr_bin_roygbiv @TechMDAI @net_termina @itsjustmarky @u1tra_instinct @QuixiAI @pupposandro Amazing list brotha thanks for sharing. Even reminded me to hit follow on people I've already seen a lot

[CocaKova @Jonnykov]

@loktar00 Waaittt a second I've seen this before. This is the one where the chunks wouldn't load! Lemme give ya a follow

[CocaKova @Jonnykov]

@MrAndyNgo Yep that's enough news for me today

[CocaKova @Jonnykov]

What did Qwen do to my childhood hero😭🤣

1x DGX Spark - Qwen 3.8 Flash Next

Prompt: "Given the RAM constraint on the Spark, how would you handle making a Sonic the Hedgehog fan game? This would be one level to test with the most beloved mechanics from the Sonic series. How would you do the art? Of course, we could make it in Godot. Don't start, but just figure out how you would handle it all with the RAM constraint"

(After I just said "proceed")

Tools: Aseprite + Godot

[CocaKova @Jonnykov]

@jun_song How do I put this in the nicest way possible... That post + comments is brimming with idiocy

[CocaKova @Jonnykov]

I'm not sure your setup, but I'll speak as if it is limited at 16GB

That rules out the use of vLLM. For some context, vLLM is greedy for the memory which puts a hard cap on memory usage. It will take the full model + cache required for the model and context. Making this not a suitable option for lower end systems.

That said, what could work for you - llama.cpp! Specifically, the dynamic GGUF quants from unsloth. Llama.cpp will load the weights only at first and as context grows - so will memory usage.

Now... Qwen 3.8 27b (to my knowledge) can be run on 16GB of memory (preferably vram of course) via the IQ4_XS dynamic quant from unsloth. Without spilling into system memory, you would have a small context window of 4k-8k from my searches...this is due to the architecture Qwen uses, BUT it runs.

I personally haven't tried it...Mistral small 24b. it's certainly not on intelligence level or qwen 3.8 (which supposedly matches GPT-4o), but it's said you can fit Mistral at Q4_K_M with a 32k context window. The model architecture and intellect differ greatly though people say.

I know it's a tough squeeze and a lot of info, but I hope I was able to help out even just a little bit.

[CocaKova @Jonnykov]

Correct! VPN via tailscale. Keryx has two modes to choose from:
1. A matrix server connection (which isn't too well known but it's essentially a self hosted chatroom; similar protocal for something like telegram/discord)
2. A direct Hermes Agent gateway connection (which is how you get the shelved sessions)

I should note I need to update my pinned post as it looks much much cleaner now.

[CocaKova @Jonnykov]

Big Keryx update on my phone tonight. 2.10.

- The sidebar finally makes sense. Sessions on shelves by day, and the folds stick.

- Filters for what needs me, what's running, what I haven't read.

- Project sessions stay in their project. A "show older" row gets me back to chats from weeks ago.

- One search box for everything: sessions, commands, settings.

- Settings are flat and searchable. Type "vibrate", it takes you to the row.

- When a turn fails it tells me what broke and offers a retry.

- Tap the context ring to see what's eating the window.

- Long-press the last reply to take the whole exchange back.

- New sessions open on the model I last picked.

- Chats open instantly. Like, before the gateway has even answered.

All of it against a Hermes gateway on hardware I own. That part doesn't change.

[CocaKova @Jonnykov]

@Kurcide Holy mackerel

[CocaKova @Jonnykov]

@SpaceX Me looking at my girlfriend listening to this lady says "Of course I still love you" 😂😂

[CocaKova @Jonnykov]

@RyanEls4 The question is, does the engineer ALSO use AI? 🤔

[CocaKova @Jonnykov]

@kawandanet @0xSero In a good way or bad way. This confuses me a little 😅

[CocaKova @Jonnykov]

@MiaAI_lab It's pure rage bait, though it's surprising to see the commenters truly believing it. I had to absolutely SCHOOL one..

sadly it's deeply nested in there

[CocaKova @Jonnykov]

@meta_x_ai @SamJWasserman @mattshumer_ Shhhh.... It's okay to be wrong

[CocaKova @Jonnykov]

I can't believe someone wrote this and mid way through thought.. "yeah, this is a good idea" and continued on to finish and publish.

[Daniel B. - AI & Tech @danielbitpro]

Tell me you don't want tech progress without telling me you don't want tech progress...

[CocaKova @Jonnykov]

@HermesWatcher Opus 5.5/Fable for the sub and GLM 5.3 flash for the local. Basic answer? yes. Major power? Also yes!

[CocaKova @Jonnykov]

@MiaAI_lab I bet they miss their muskets too with that thinking