Carlo

On my own journey towards financial literacy and freedom | Travel planner & Bike guide | 🇮🇹 🇳🇱

[Carlo @CarloFCesar]

@Zai_org identity crisis? 😅

[Carlo @CarloFCesar]

@witcheer The book goes from math to behaviorism, psychology, philosophy and ends up at AI. Not an easy read, but a fascinating approach (2026)

[Carlo @CarloFCesar]

@b_nnett Wow man, nice! Compliments on the build

[Carlo @CarloFCesar]

I calculated what I’d need more or less to go full local with my project and I’d need 2 @NVIDIAAI DGX sparks 🤑💰
For now 24GB ram have to do haha! But I also believe local models will become smaller, better and perhaps even a commodity. The apps we built and top (output) and the energy that goes in (input) might become the true value!

[Carlo @CarloFCesar]

@AlexFinn Went there from Openclaw, started my project from scratch. First 48h left a great impression!

[Carlo @CarloFCesar]

@elroyic @AlexFinn Why not 70B, could quantize FP8 or NVFP4 even. And then, how does QWEN relate to for example Llama?

[Carlo @CarloFCesar]

@luminousmind_co @AlexFinn That's a great addition, thanks @luminousmind_co

[Carlo @CarloFCesar]

@E_Jellerson @CardilloSamuel @VadimStrizheus Yup… lessons learned

[Carlo @CarloFCesar]

What would you run if your have 2x DGX Spark (and 1x mac mini 24gb…)and want to go 100% local? Multi agent system that needs to 24/7 run computations and cross-references on an ever growing vectorized data pool. Nemoclaw strongly preferred.

- Yes I’m relatively new to it
- Yes I first got the mac mini and learned a lot (hence the upgrade)
- Yes I need some help here. 😂

[Carlo @CarloFCesar]

If you don’t read up on the updates on AI for more than 24hours you feel sooooo behind

[Carlo @CarloFCesar]

My lesson for today: ''Sleeping turns knowledge into wisdoms''

Not a 1 on 1 quote, but it's the concept.
Comes from:
https://t.co/AI1sAKXQd2

[Carlo @CarloFCesar]

@sidelined_cap Agree there man! I use Qwen 3.6 via openrouter only to do online peer reviewed research for me

[Carlo @CarloFCesar]

@witcheer @AlexFinn I just added Qwen3.6-plus-preview for non sensitive prompts and cross (research pulls in my case), previously I ran these on Haiku or Sonnet. Moving on! https://t.co/qs12WaN7lx

[Carlo @CarloFCesar]

Just cut my agent's cloud costs significantly without sacrificing quality.

Thanks to @witcheer and @AlexFinn, I starting building from your posts!

The journey:

Started on Mac mini M4 (24GB…) running Claude Haiku/Sonnet for all background tasks ~60 API calls/day.

Tried Qwen3.5-35B-A3B via Ollama first. 23GB model. OOM. Killed.
Tried Qwen2.5-14B. Fit fine, but reasoning quality too weak for my workflow.

Then TurboQuant dropped @GoogleResearch. KV cache compression, 4.6× smaller memory footprint!

My new stack:
Qwen3.5-27B-IQ3_XXS (10.7GB) via llama.cpp TurboQuant fork → 13.6 tok/s on M4 Pro → zero API cost.

Validation before shipping:
→ 15 tool-use scenarios: 12/12 pass
→ Shadow test against Haiku on live data: output indistinguishable
→ Did have 3 timeout failures on error recovery — but a lot of crons can run overnight with a higher timeout, not a quality issue

My full model stack today:

🟢 Local (free)
→ Qwen3.5-27B IQ3_XXS — health monitoring, training load, environmental logging. (Based on multimodal biomarkers, keen on getting that data vectorized!)
→ MedGemma 4B — offline domain-specific model

🔵 Anthropic
→ Claude Sonnet — interactive sessions, memory synthesis
→ Claude Haiku — research briefings, clinical alerts, complex reasoning chains

🟡 Google
→ Gemini 2.5 Pro — fallback when Anthropic unavailable

⚫ DeepSeek
→ DeepSeek V3.2 — cost-efficient tasks when applicable

The "local = cheap but dumb" assumption is breaking down fast fast faster!

Probably it is already outdated, goes so fast! Curious to get it smarter and cheaper everyday 🥳

[Carlo @CarloFCesar]

@VadimStrizheus Ok. This already changed.... just added Qwen3.6-plus-preview (free tier). Nothing sensitive or private. I runs my research crons and prompts through it now instead of Sonnet 4.6
https://t.co/qs12WaN7lx

Things move fassssst

[Carlo @CarloFCesar]

@VadimStrizheus I’d do Qwen 27, not 35 to be honest. With 35 you won’t have enough headspace. And run turboquant on top to compress kv chache. Then make sure to have some cloud api for when you really need it

[Carlo @CarloFCesar]

Things really do go fast!
Just set all my online research cross and prompts to run on Qwen3.6 Plus Preview (free tier) on OpenRouter.

No personal data, no MEMORY.md, no strategy, nothing sensitive because Alibaba collects prompts on the free tier.

Check it out here: https://t.co/0lfGTovTtw

[Carlo @CarloFCesar]

Just cut my agent's cloud costs significantly without sacrificing quality.

Thanks to @witcheer and @AlexFinn, I starting building from your posts!

The journey:

Started on Mac mini M4 (24GB…) running Claude Haiku/Sonnet for all background tasks ~60 API calls/day.

Tried Qwen3.5-35B-A3B via Ollama first. 23GB model. OOM. Killed.
Tried Qwen2.5-14B. Fit fine, but reasoning quality too weak for my workflow.

Then TurboQuant dropped @GoogleResearch. KV cache compression, 4.6× smaller memory footprint!

My new stack:
Qwen3.5-27B-IQ3_XXS (10.7GB) via llama.cpp TurboQuant fork → 13.6 tok/s on M4 Pro → zero API cost.

Validation before shipping:
→ 15 tool-use scenarios: 12/12 pass
→ Shadow test against Haiku on live data: output indistinguishable
→ Did have 3 timeout failures on error recovery — but a lot of crons can run overnight with a higher timeout, not a quality issue

My full model stack today:

🟢 Local (free)
→ Qwen3.5-27B IQ3_XXS — health monitoring, training load, environmental logging. (Based on multimodal biomarkers, keen on getting that data vectorized!)
→ MedGemma 4B — offline domain-specific model

🔵 Anthropic
→ Claude Sonnet — interactive sessions, memory synthesis
→ Claude Haiku — research briefings, clinical alerts, complex reasoning chains

🟡 Google
→ Gemini 2.5 Pro — fallback when Anthropic unavailable

⚫ DeepSeek
→ DeepSeek V3.2 — cost-efficient tasks when applicable

The "local = cheap but dumb" assumption is breaking down fast fast faster!

Probably it is already outdated, goes so fast! Curious to get it smarter and cheaper everyday 🥳

[Carlo @CarloFCesar]

@no_stp_on_snek @VadimStrizheus Exactly this ye!