Why I went all-in on Local AI.
Not to save money. Two GPUs and a Blackwell box will never pay themselves back at $20/month.
I did it because I’ve spent 25 years watching people get locked out of systems they don’t own.
@loktar00@AutismCapital I think the tension between open source and the big labs is gaining enough momentum that we might actually be able to make a difference this time.
I love the photos of all of the home data centers, they are all beautiful to me. I love the fact the the true judgement point is "does it work, and what cool shit are you doing with it?" not how sleek and clean it looks. Sometimes progress is messy and enthusiasm beats sleek lines at this point in the game. Keep building!
Quick Headups. You DONT NOT Need Breakaway Cables to Run 4 Sparks on a 812 Switch and possible a 804. 400G Ports are Backwards Compatible. We not waiting til tomorrow we going tonight 🙏🏽🔥
Using AI to enhance or clarify, your thoughts is the point of the tool. It’s a huge unlock and there’s a difference between what you’re doing and AI generated slop. There are real insights here and I have learned so much for you. Don’t let your detractors slow you down at all. Keep it up.
So many people bash the @NVIDIAAI DGX Sparks for having low bandwidth.
While that's true, one thing I understood while studying what hardware to buy is that no hardware is perfect, you gotta sacrifice something.
Want raw speed? Get GPUs, but you are restricted on VRAM (which is used to load the models on)
Want to serve large models? Go for unified memory, i.e. a Spark, Mac or these new AMDs, but then you sacrifice bandwidth (which is used for generating tokens, higher is better)
Then you gotta consider energy consumption, cooling, noise, form factor etc..
Pick your poison.
@kaaaash____ I have a touchscreen windows machine and it mostly just messes me up. So no, I would not. I like my MacBook just like it is bring back the touch bar if you wanna do something.
Just moved my Hermes setup to a new rig. I've never dealt with a smoother transfer with literally anything. If I copy 2 lines of code something breaks.
@NousResearch what are you on
At one point I was doubting myself thinking this is too good to be true.