Random Libertarian Tech Lead

Shipping to prod in a $100M+ ARR unicorn startup w/ local AI.

4x DGX Spark (GLM 5.3 Flash), 1x M5 Max 128GB (Qwen 3.8 Flash Next), 1x RTX 5090 (Qwen 3.8 27B).

[Random Libertarian Tech Lead @someRandomDev5]

@jmacftw The knowledge of how to do it, and some public datasets from Hugging Face.

[Random Libertarian Tech Lead @someRandomDev5]

@FT_IOxCS GLM 5.3 is pretty slow on 4 Sparks. Honestly, it's more like an 8-Spark model.

It's kinda like how GLM 5.3 Flash can be used on 2 Sparks but doesn't get to a reasonable speed until you have 4.

[Random Libertarian Tech Lead @someRandomDev5]

@jmacftw Historically for a lot of people, the natural pushback to fine-tuning a classifier model for this (or building one from scratch) is that they didn't have the expertise. But if a big smart model is gonna do it for you, that barrier evaporates.

[Random Libertarian Tech Lead @someRandomDev5]

@jmacftw The only way to tell would be to compare upwards from a smaller fine-tuned domain-specific model. Have you tried that?

[Random Libertarian Tech Lead @someRandomDev5]

@ThePrimeagen Currently working on an "intent authoring" prototype PoC I plan to use at work, meant to flesh out the planning phase of a ticket alongside AI, where the interaction design is a mix of "mind map" and "whiteboard" (but with local asynchronous AI interacting frequently).

[Random Libertarian Tech Lead @someRandomDev5]

This is awesome! We are still so early in rethinking how we interact with agents. It's definitely important to keep an open mind, and be willing to ideate and explore with novel ideas. I've decided to start saying "intent authoring" instead of "prompt engineering" in order to think outside of the (literal textbox) box.

[Random Libertarian Tech Lead @someRandomDev5]

@sertlock @ThePrimeagen You know yourself well, sert.

[Random Libertarian Tech Lead @someRandomDev5]

@theo @lovesosaa888 Local AI's biggest haters are the ones who can't afford the hardware, and have to rationalize themselves into hating it so that they can cope with being unable to use it.

Go read Aesop's "The Fox and the Grapes" (the origin of the term "sour grapes").

[Random Libertarian Tech Lead @someRandomDev5]

@theo @lovesosaa888 Not a hypothetical analogy btw; It happens often, and it's literally happening right now.

[Random Libertarian Tech Lead @someRandomDev5]

@theo @ThePrimeagen Claude is down quite often. One of many reasons strong local AI makes sense.

[Random Libertarian Tech Lead @someRandomDev5]

@theo @Conor_D_Dart @jagrat_7 > You reply to my posts without following me [...]

I can do one better: I reply to your posts despite having you muted. Lmao.

What an annoying passive-aggressive way to implicitly beg for followers. It's like a whiny 12-year-old boy.

[Random Libertarian Tech Lead @someRandomDev5]

@theo @dmarcosm98 Too bad for you that you unjustifiably hate local AI, then. It resolves those problems perfectly.

[Random Libertarian Tech Lead @someRandomDev5]

@shivjam Eventually since there's no meaningful foundation to be upset by generative AI at all, all of the complaints against it will erode and collapse. Feeling like you had a good reason to be mad about generative AI was all an illusion.

[Random Libertarian Tech Lead @someRandomDev5]

@shivjam I'm so happy to see people with unjustified arbitrary principles cannibalizing themselves. When you have ridiculous principles and nobody can find a good justification for why those principles exist, everybody draws the lines at weird arbitrary positions and nobody agrees.