Artem Mats

Coding with Local AI and LLMs: 2 x DGX Spark cluster experiments.

[Artem Mats @Reederey]

GLM-5.3 Flash is my daily driver on 2× DGX Spark. I prefer NVIDIA’s NVFP4-Local weights to DeepSeek v4.1 EXL3 quants.

1M context, ~45 tok/s single-stream, auto-start after reboot. Will work on that further.

My setup + benchmarks:
https://t.co/WI71OdIWxH

[Artem Mats @Reederey]

@christopherowen I can help with pleasure, as I am working on a very similar topic.

[Artem Mats @Reederey]

Mini or Duo for mobile coding?

Prices are very close:
iPad mini Wi‑Fi ($699) + iPhone 18 Pro Max ($1,299) = $1,998.
iPhone Duo starts at $1,999.

My preference: mini + phone for longer reviews; Duo for the convenience of one pocketable device.

[Artem Mats @Reederey]

Inkling 2 small weight will be released soon!!!

[Thinking Machines @thinkymachines]

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.

Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.

https://t.co/VR02tw73fJ

[Artem Mats @Reederey]

@SamuraiPhonk Apply the codex plugin within the app without any special setup.

[Artem Mats @Reederey]

I switched from Droid CLI and Hermes Agent to DeepSeek Harness for day-to-day coding.

Quantized GLM-5.3-Flash runs well on my NVIDIA DGX Sparks, and I can call in Codex to help plan a change or give the code a final review.

The plugin catalog makes it easy to add MCP tools, browser automation, and reusable coding workflows as I need them.

https://t.co/jnBSdQ5b4a

#DeepSeekHarness #LocalAI #DGXSpark #Codex

[Artem Mats @Reederey]

New CUDA Kernel King came!!!

[Kimi.ai @Kimi_Moonshot]

Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.

🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
🔹 Full-project coding: games in Unreal, Unity & Godot.
🔹 Excels at parametric 3D modeling in Blender & CAD.
🔹 More reliable at finance work: Accurate modeling with hallucination down by 65%.
🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.

Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on https://t.co/85jA71X9gw, Kimi Work, Kimi Code, and the Kimi API.

Open weights by Nov 4, 2026.

🔗 API: https://t.co/XCrgjXAqMw
🔗 Tech blog: https://t.co/5iPP0MBej6

[Artem Mats @Reederey]

@JannerChang Just follow
https://t.co/9Innakopq0

[Artem Mats @Reederey]

This is why NVIDIA’s QAD quants matter for local AI.

GLM-5.3-Flash-FP4-Local gets extra training to recover what compression loses. With quantization-aware distillation (QAD), the original BF16 model acts as a teacher, helping the compressed model learn to give similar answers while working with lower-precision weights.

That takes more work than a regular quant conversion: NVIDIA used 500 million training tokens over 3 epochs for this release.

The result is a 320B model using about 166 GiB, validated on two 128 GB DGX Sparks, with room for context. Quality stays close to BF16 on NVIDIA’s two reported benchmarks: GPQA 91.3% vs 92.1%; tool use 92.4% vs 91.9%.

EXL3 at 2–3 bits per weight can squeeze models into even less memory. But fitting a bigger model doesn’t automatically mean better answers. For my coding and math work, this QAD GLM build is a much better choice than the DeepSeek-V4.1-Flash 2-3 bpw EXL3 builds I tested on the same two Sparks.

I’d rather preserve more of a model’s ability than chase parameter count at the lowest possible bitrate.

More releases like this, please @NVIDIAAI !!!!

Model: https://t.co/QiVpRSiiYH
How QAD works: https://t.co/bJse7AIBPz

[Artem Mats @Reederey]

I have tested nvidia/GLM-5.3-Flash-FP4-Local and one of the best DSV4.1 Flash EXL3 quads of the LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw on a 2 x DGX Spark cluster on real coding and math problems. Better output was given by the QAD NVFP4 quants of the GLM 5.3 Flash distilled by NVIDIA. No looping, better instruction following, working fine in Deepseek-Harness. It is my #1 choice for Local AI on 2 DGX Spark machines now.

Thanks @NVIDIAAI for supporting local AI.

[Artem Mats @Reederey]

QAD - It is what NVIDIA did. The distilled 175B nvidia/GLM-5.3-Flash-FP4-Local model fits 2 Sparks and offers 2M tokens of FP8 KV Cache with minimal accuracy loss. It has run smoothly for two days. However, quantized DS v4.1 Flash EXL3 2–2.8 weights on a difficult tasks model eventually loop on “Hmm” once context fills. QAD seems best for managing weights on Sparks 2 x 128GB.

[Artem Mats @Reederey]

Newest cut of DeepSeek v4.1 Flash has arrived with stronger banch results.

[DeepSeek @deepseek_ai]

🚀 DeepSeek-V4.1-Flash-1017 is here!

Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.

🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.

1/4

[Artem Mats @Reederey]

Claude Haiku 5.5 Has Been Released

Nearly a year after the prior release, Anthropic has chosen to enter the budget tier. Benchmark data indicate a clear edge over GPT-6 Luna. Costs are $0.1/$0.5 per million tokens for inputs up to 100k tokens; beyond that length, rates rise fivefold to $0.5/$2.5.

[Artem Mats @Reederey]

@sfxnz API’s DeepSeek shows a lot of “Hmm” looping; how are 2bpw quants going?

[Artem Mats @Reederey]

@sfxnz Why is every post limited to performance parameters, with no one sharing results from applied quants on the Terminalbanch or Deepswe?

[Artem Mats @Reederey]

🚀 v1.6.0 is out �� GLM-5.3-Flash EXL3 on 2x DGX Spark

35 commits since previous release.

What moved:
• Decode: prose ~30.8 → ~34.8, structured 69 → ~75 @ full 7/7, essay → ~27.4
• Agents stop re-reading history: 110k replay ~98% (4s not 132s), 4x60k sessions 98.7%
• Ops hardened: dual-HCA, 50ms SHM recheck baked in, prefix admission + resident tail

Why not TensorFold, I know about decode speed. But I optimized it for: agentic coding, tool usage, and re-reading speed of long histories in a 1M window on a stable vLLM stack.

More info here:
https://t.co/4619ztNkrs

[Artem Mats @Reederey]

@0xSero Did you measure perplexity, or have run DeepSWE banch on this quants?

[Artem Mats @Reederey]

OpenAI is lowering the limits for its $200 Pro subscription.

At the same time, they are reopening sign-ups for this tier. They justify the move by noting that the total output capacity will still exceed what the Pro subscription offered a month ago (prior to the Astra release) and by promising not to bring back the five-hour limits. Additionally, they promise to add new features to the subscription that won't count against usage limits.

All this is happening against the backdrop of the cancelled Astra 6.1 release—due to safety concerns regarding the model—and rumors of an upcoming $500 "Pro Max" plan. Meanwhile, their main competitor, Anthropic, has just launched the highly successful Opus 5.5 and Sonnet 5.5 models.

[Artem Mats @Reederey]

👀