gianfranco

Inference Engineering Manager of @QVAC at @Tether | Not your AI not your Intelligence | Views are my own

[gianfranco @gianniCor]

We have added Cluster inference support to Qvac Fabric. You can cluster together multiple machines and split the inference across them. Supporting both tensor and layer parallelism. Here 2x DGX Spark running Deepseek-v4-flash-0731 over RDMA.
https://t.co/FQkHF1e8Fg

[gianfranco @gianniCor]

Latest @qvac Fabric work added CoreML support to make Parakeet insanely fast. You can transcribe 25 mins audio in nearly 16 seconds on an Iphone 16.
check https://t.co/c3ZZzg8uci

[QVAC @qvac]

QVAC Fabric now runs Parakeet's encoder through Core ML on Apple silicon, which moves it from the GPU to the Neural Engine.

On a MacBook Air M2, Parakeet TDT 0.6B v3 went from 78x real time on the Metal GPU to 164x through Core ML.

[QVAC @qvac]

Parakeet Redux is compressed from NVIDIA's Parakeet TDT 0.6B v3, so we ran it against the original, which QVAC runs in Q8.

Both transcribed the same two hours of audio on the GPU of a MacBook Pro M5 Max.

QVAC took 22.3 seconds, 323x real time, and Redux took 44.2 seconds, 163x. Both times include loading the model.

[QVAC @qvac]

QVAC SDK 0.20 is live. 🚀

We added support for:

- ABot-World: generates a walkable world from a single photo
- MiniMax-H3: create video with synchronised soundtrack
- Video models can now run on GPUs with less VRAM
- TranslatePsy-AfriSLM: translate between English & 19 African languages on-device
- ACE-Step can now generate entire songs from a single sentence & more new features

We also merged the latest changes in llama.cpp into QVAC Fabric, to ensure continuous compatibility.

Learn more in the thread below.

Complete release doc here: https://t.co/N8hcETdJHF

[QVAC @qvac]

The QVAC audio stack is continuously optimised. We compared its performance against several other open-source solutions, and QVAC comes out fastest on transcription, speech synthesis and music generation, across every GPU lane we measured.

If you want to build fast apps with local audio AI that fit on consumer devices, you should definitely check QVAC out.

https://t.co/3AcGpL3QRE

[gianfranco @gianniCor]

Did you check latest inference optimizations on @qvac for speech models ?

[QVAC @qvac]

https://t.co/7gJGFdcGG8

[QVAC @qvac]

https://t.co/7gJGFdcGG8

[gianfranco @gianniCor]

@dhh what about GPU acceleration on Mac?

[gianfranco @gianniCor]

@TeksEdge what about vulkan support?

[gianfranco @gianniCor]

@RaffoMancio @deepseek_ai Nearly 19 tps during generation, unfortunately the video was cut cause too long

[gianfranco @gianniCor]

Deepseek v4 flash is now available on Fabric. We added Vulkan and Metal support, so you can run it on any gpu. Performance improvements come from optimized fused kernels. Go and try the new @deepseek_ai released version '0731' on https://t.co/OzCiML7whP

[gianfranco @gianniCor]

@HugoSCM @deepseek_ai Q2 as it is the one that fits best in the Strix

[gianfranco @gianniCor]

@deepseek_ai Demo is on AMD strix halo

[gianfranco @gianniCor]

@antirez cooking something similar for NVidia and Metal as well

[gianfranco @gianniCor]

DeepSeek V4 Flash on Strix Halo just got a major Vulkan boost on Fabric.
Fusing different ops improved prefill way more than vanilla llama.cpp, closing the gap with @antirez ds4 ROCm implementation.
Coming soon to https://t.co/FQkHF1e8Fg.

[gianfranco @gianniCor]

@antirez love it. straight to the point

[gianfranco @gianniCor]

@antirez what are the performances for deepseek v4 flash ?

[Marco Chiappetta @elchiapp]

🚢

[gianfranco @gianniCor]

@no_stp_on_snek @digitalix Take it easy man 😂 just noting the differences. no QJL support and no Vulkan support on your repo, simple as that

[gianfranco @gianniCor]

@no_stp_on_snek @digitalix in that repo there is real turboQuant implementation, polarQuant + QJL step