Costanzo Pad

Bioinformatician 🧬 | Local AI & LLMs | Scientific workflows, genomics and AI-assisted research | Testing what actually works

[Costanzo Pad @CostanzoPad]

⚡ Qwen3.8-27B DFlash2 NVFP4 GGUF is live.

1.02 GiB speculative draft • 49 NVFP4 tensors • Apache-2.0 • SHA-256 verified.

Model:
https://t.co/Qye8y05Km7

QVIR-1 R2 code + 150K profile:
https://t.co/B8D5V0PWV5

#DFlash2 #GGUF

[Costanzo Pad @CostanzoPad]

🚀 Qwen3.8-27B NVFP4/Q8 Hybrid Analytical GGUF is live on Hugging Face.

20.08 GiB • 168 NVFP4 + 338 Q8_0 tensors • F16 vision projector • BioAgent 100/100 • full QVIR-1 R2 provenance.

https://t.co/JtIORGWkf3

#GGUF #llamacpp

[Costanzo Pad @CostanzoPad]

🧠 Qwen: “Wait… let me think a bit more...”
⏳ 10,000 tokens later
🤔 Qwen: “I think this is correct.”

🔧 Tool: “Or… you could just verify it.”

What if that’s the real point?

Maybe the answer isn’t making Qwen reason more.
Maybe it’s forcing Qwen to use tools whenever something can be checked:

📂 inspect the file
💻 run the code
🧮 compute the value
📘 check the documentation
✅ verify the output

Basically:

think less → use tools → verify → answer

For many real-world tasks, Qwen LOW + tool verification might actually be far more reliable than Qwen HIGH reasoning on its own.

[Costanzo Pad @CostanzoPad]

Can a local 27B vision model debug scientific work?

Qwen3.8-27B + QVIR-1 built and verified a ComplexHeatmap on 2× RTX 5060 Ti—recovering from 3 R errors and reaching 92.6k context.

26.8 tok/s decode. 54.7% DFlash2 acceptance.

Full experiment ↓
https://t.co/CdCOqy93l0

[Costanzo Pad @CostanzoPad]

New stealth model just dropped: #Ox #Alpha on OpenRouter.
1M context • multimodal (text/image/video) • free for a week.

Some say it’s small… but fingerprints scream Zhipu GLM multimodal and early results look frontier. Everyone’s excited. Go test it before the free week ends.

[Costanzo Pad @CostanzoPad]

A lot of the “Qwen is too slow / overthinks everything” experience may actually come from the reasoning template and default configuration, rather than from the model itself.

[Costanzo Pad @CostanzoPad]

I think an important point for people testing
#qwen 3.8-27B is the reasoning template. If you want the model to feel interactive, set reasoning_effort to LOW.

[Costanzo Pad @CostanzoPad]

So, roughly:

LOW → interactive daily work
MEDIUM → complex tasks
XHIGH → deliberate deep reasoning

[Costanzo Pad @CostanzoPad]

So, roughly:

LOW → interactive daily work MEDIUM → complex tasks XHIGH → deliberate deep reasoning
A lot of the “Qwen is too slow / overthinks everything” experience may actually come from the reasoning template and default configuration, rather than from the model itself.

[Costanzo Pad @CostanzoPad]

Then treat MEDIUM/XHIGH as escalation modes, not as the default: use them when the agent actually needs to solve a complex, multi-step problem.

[Costanzo Pad @CostanzoPad]

I just published Can a Local 27B Vision-Language Model Debug Its Own Scientific Visualization? https://t.co/Yy3CcYl8bM

[Costanzo Pad @CostanzoPad]

Very interesting model! thanks @Alibaba_Qwen @UnslothAI

[Costanzo Pad @CostanzoPad]

@thsottiaux Easy question: Whatever I want to build, I can now do it thanks to Codex.

[Costanzo Pad @CostanzoPad]

Maybe for scientific infographics we should rely less on AI video generation and more on agents like Codex connected via MCP to scientific and graphics software.

The result would be less “plausible-looking” and far more deterministic, reproducible, and grounded in real data.

[Costanzo Pad @CostanzoPad]

@haydendevs The question should be relative to speed you want. In my cases, I use 2 x RTX 5060 ti 16 GB + 64gb RAM + M2 2tb 7gb. I use deepseek in streaming and I got 103-80 tok/s prefill and 3-5 tok/s decode. It Is slow, I know. But I closed a lot of works in a day.

[Costanzo Pad @CostanzoPad]

🚀 New repo: deepseek-v4-local

Running DeepSeek V4 locally on a consumer PC with 2× RTX 5060 Ti 16GB, 64GB RAM and NVMe expert streaming.

~5.8 tok/s at 96K context.

Repo 👇
https://t.co/ooRIMTCI8p

#LocalLLM #DeepSeek

[Costanzo Pad @CostanzoPad]

4/ What makes this interesting

This is not only a benchmark.

The model runs through an authenticated OpenAI-compatible server and has completed real OpenCode sessions with tool use and multiple consecutive requests.

The project is still experimental.

[Costanzo Pad @CostanzoPad]

I managed to run Antirez’s DeepSeek V4 Flash locally on a consumer PC:

• 2× RTX 5060 Ti 16 GB
• 64 GB RAM
• NVMe SSD
• Windows + WSL2
• no NVLink or GPU P2P
The model is ~80.8 GiB, so it cannot fit entirely in RAM or VRAM.

[Costanzo Pad @CostanzoPad]

3/ What improved performance

Using two 64 MiB pinned buffers per GPU allowed SSD reads and host-to-device transfers to overlap.

Prefill improved from:

92.70 tok/s → 103.34 tok/s

That is an 11.48% gain.