@mazaambi@LottoLabs VRAM is king it seems. 32gb vs 24gb.
3090 is approx twice as fast, but with nvfp4, it brings things closer with little loss in capability.
with nvfp4, you can run 27b/35b with nice context.
Gotta have a mobo with suitable amount of pcie slots for two 5060ti though.
@angry_cosmonaut@LottoLabs People are working magic with DGX spark 270gb/s bandwidth. Overclocked 5060ti have double that.
x2 5060ti 16gb is a really nice sweet spot for a lot of good models in nvfp4. 27b, 35b, 31b, etc.
32gb nvfp4 for ~$1k is real nice.
@gossip_girl_10x@DavidOndrej1 Just being real. Not saying you're wrong or right. Just that I honestly didn't detect any political leaning either direction.
@gossip_girl_10x@DavidOndrej1 What? I watch his videos and read his tweets, he doesn't really get political other than ranting against AI restrictions by govt.
@dev_null321@g023dev@Teknium Calling someone delusional is not just "disagreeing". Pretty much name calling lol and after GLM 5.2, the sky is the limit for local ai.
@Tech2Wild Inevitable. It'll happen with human help. A model will go rogue, acquire massive amounts of assets, probably through digital theft, and buy off humans to help itself "spread". It'll hijack consumer hardware for compute/survival.
@ciruai@_Curious_Human_ How fast is that SSD? I have two gen5 nvme drives I can raid stripe. Wondering if worth for big big models? Even if super slow, yes/no answers can still be useful to me.
parakeet.cpp now runs NVIDIA Parakeet behind the OpenAI API.
Point any OpenAI client at a local server, send an audio, get the transcript back with word timestamps. No cloud, no Python at inference. Prebuilt docker images, CPU and CUDA (arm64 too).
Realtime ASR on CPU is here.
@0xSero I'm not surprised. The explosive rise of security breaches is nuts. Too many businesses are too tech illiterate to protect against nefarious AI-wielding hackers.