@RobertFreundLaw There is a few “cluely” style apps that can do that chunks dialogue and runs it against a rag. Seems like a good use case, i think filevine is trying to roll it out
It was the first thing i bought thinking like you are. Then Just realized how slow it is, for that price can do a 5090 computer and run 27b dense and then use cloud models sparingly. Or mac studio with slower but larger models. Its amazing device but usuable for realtime is not of them.
@ivanfioravanti@alexocheema@exolabs hrmm, I also have a M4 Max 36gb which i thought was not consequential to run, but would running that to get to 4 help -- or should i just stick with the studios?
Ok mlx + RDNA + @exolabs
lets crack these open an see what we can run.
M5 Max 128 MBP / M3 Ultra 256 / M3 Ultra 512.
What models should I try?
Any things i need to know?
@alexocheema
Local model guys...
I think I have to join your cult now. Please accept this offering:
Figured out how to get a Dell T2 Workstation + RTX PRO 6000 96GB for less than the cost of the card alone.
The trick:
• Platinum Biz Amex + Dell program = $1,150 back on $5k purchases
• Stacks with Dell's 5% offer (if you have the offer)
• Split across 2 Amex cards = up to $2,700 total
Bonus: Dell Rewards on top = another $300–600
You're not buying a GPU. You're getting paid to buy a GPU.
Local model guys...
I think I have to join your cult now. Please accept this offering:
Figured out how to get a Dell T2 Workstation + RTX PRO 6000 96GB for less than the cost of the card alone.
The trick:
• Platinum Biz Amex + Dell program = $1,150 back on $5k purchases
• Stacks with Dell's 5% offer (if you have the offer)
• Split across 2 Amex cards = up to $2,700 total
Bonus: Dell Rewards on top = another $300–600
You're not buying a GPU. You're getting paid to buy a GPU.