Aleksandar Janca

I help companies run on autopilot in 7 days. co-founder @ &

[Aleksandar Janca @datachad]

The question that kills most AI projects in week one:

"Where does the data live?"

Usually: three spreadsheets, one inbox and someone's head.

[Aleksandar Janca @datachad]

@jun_song a chat client using more memory than docker is a good reminder that the boring bugs need an owner too, the model was never the whole product

[Aleksandar Janca @datachad]

@TechWithLilyAi the travel bot booking flights while he sleeps is the one I would gate, anything that spends money waits for a yes from a person

[Aleksandar Janca @datachad]

@PaulGug @liquidai @OpenBMB @SparkLLM @PrismML @NVIDIAAI all rounder is the part I would drop, at that size each one earns its place on one narrow job like routing tickets or pulling fields out of an email

[Aleksandar Janca @datachad]

@AntLingAGI @NVIDIAAI no detectable accuracy change on gsm8k is the part I would rerun on tool calling, short math answers hide what fp8 does to long structured output

[Aleksandar Janca @datachad]

@0xWhiteMage one box, clustering looks cheap until you spend a weekend debugging the link between them instead of using the model

[Aleksandar Janca @datachad]

@Morrowmake bandwidth for decode sure, but with agents the prefill on a long context is what I end up waiting on, and thats compute

[Aleksandar Janca @datachad]

@ivanfioravanti @fabien_martin_ true, three agents running in parallel are three users as far as the machine cares, and thats exactly the case the spark numbers are built for

[Aleksandar Janca @datachad]

@10gynn experts offloaded to nvme is clever, I would just measure what it does to the first token once that cache goes cold

[Aleksandar Janca @datachad]

@giyu_codes fair, I would still add an alert whenever it takes a different path than usual, a silent workaround on a vendor portal is the one that scares me

[Aleksandar Janca @datachad]

@NashBriggsy monitors being the only repeat payers is good to know this early, the first agent that comes back for real work is the week worth writing up

[Aleksandar Janca @datachad]

@wineplacent @sudoingX dropping vision to buy context is a trade I would make for most agent work, they rarely need to look at anything

[Aleksandar Janca @datachad]

@Blackwellboy @andersjw_ @bertholomusai respect, a number taken at 128k is the only one that matches what an agent actually does to the machine

[Aleksandar Janca @datachad]

@otrenav exactly, and often the process change alone gets most of the gain, the model ends up with a much smaller job than planned

[Aleksandar Janca @datachad]

@kunal_twts at that size the saving can be real, but even there the contract clause is usually what starts the project

[Aleksandar Janca @datachad]

Self-hosting an LLM rarely saves money at small volume.

People do it because of a clause in a contract.

Valid reason. Just budget for the GPU, the updates and the person who keeps it running.

[Aleksandar Janca @datachad]

@surlymochan yep, and at a small company that pager is usually the founder, the most expensive person in the building to wake up

[Aleksandar Janca @datachad]

@Winterrose @theo defaults matter most for the mcp part, every server switched on by default is one more thing with access to the repo

[Aleksandar Janca @datachad]

@fabien_martin_ @ivanfioravanti the question I would ask first is how many people hit it at once, one user fits a mac, a whole team starts to look like the spark

[Aleksandar Janca @datachad]

@jgonzalezferrer I run a quantized qwen 9b through ollama on an m3 pro with 36gb, it sits in 14gb, decide the job before you buy the box

[Aleksandar Janca @datachad]

@manxlab learn to read logs before anything else, most of what breaks tells you exactly where in a file nobody opened