Crown 👑

Local AI Consulting
AI is about the workflow, not the model.

AMD Local LLM Group:

[Crown 👑 @ciruai]

@the_jimmy_jones https://t.co/Cscs4Wloj8

[Crown 👑 @ciruai]

@awdyzx @DataChaz Use the real one instead of this one

[Crown 👑 @ciruai]

@Hassaansaeed22 @DataChaz If you're memory poor use the new QAT version of gemma4 12b it's much faster

[Crown 👑 @ciruai]

@ddarius94 @DataChaz You might be able to run hermes with it.

[Crown 👑 @ciruai]

@MichaelGannotti White Glove Small business AI Solutions and consulting.

[Crown 👑 @ciruai]

@mr_r0b0t Have you tested?

[Crown 👑 @ciruai]

@bradmillscan @shannholmberg @hey_madni Followed both!

[Crown 👑 @ciruai]

@MichaelGannotti @NVIDIAAI The models bigger than that are not worth running anyway, you are sitting on a beast right there!

[Crown 👑 @ciruai]

@MichaelGannotti @mr_r0b0t I thought you had the same system? (hardware wise)

[Crown 👑 @ciruai]

@mr_r0b0t slop influencers.

[Crown 👑 @ciruai]

@sirius_devops @sudoingX Nah it's cheaper, but not 2k.

$3~

[Crown 👑 @ciruai]

@DBirker78883 @sudoingX Absolutely. I hope his benchmarks do it any justice at all.

[Crown 👑 @ciruai]

@sudoingX its not fair to use same quants on both when there are quants optimized for each.

[Crown 👑 @ciruai]

@SMNYC1 @arunninghacker blows my mind what even a 3GB model knows. Where does it keep it all? 😂

[Crown 👑 @ciruai]

@SMNYC1 @arunninghacker Imagine power is out for extended period and you're the only house which basically still has the whole internet self contained within it!

[Crown 👑 @ciruai]

@SMNYC1 @arunninghacker Currently solar powered and running AI locally. Hypothetically if power and internet goes out I could still run AI at home

[Crown 👑 @ciruai]

@GMMeyer @benrayfield @BLUECOW009 I have no illusions about someone's ability to just download a weight file. Specifics are going to matter. That's why I say we can't really war game it without the details.

[Crown 👑 @ciruai]

I rely heavily on openai still at home, even though I have access to dozens of models running locally and have spent hundreds of hours preparing for local only use. A couple of times in the last month the internet went out for an extended period (someone crashed into a main hub and died 🥲). Had to test everything I had been working on and keep moving without it. Found some important gaps. It's a lot of fun though, to be honest.

[Crown 👑 @ciruai]

GPU VMs are much more expensive than just using an API. It's going to cost you $1+ an hour. Adds up if it's always on. An API for chatting probably will only cost you $20 a month or less but at that point why are you not just using gpt anyway? The reason to go local is to be in control.

[Crown 👑 @ciruai]

@GMMeyer @benrayfield @BLUECOW009 Would need to dig into specifics to map out the best way. Kind of hard to say hypothetically. Impossible to stop if people can't be trusted. In the end it's your people who protect you.