Pipefitter/steamfitter lost in AI solving a problem. Crypto, models, and whatever I break this week. ๐จ๐ฆ I knows a few things, but it's mostly about nothing.
@keendisregard@muradylan@thinkymachines neat that a 26b runs offline on a 15w chip at all tbh. igpus have been a wall for me too , had high hopes for the 140t in my nuc and got really bad results
@Ubunta the "on a mac maybe" part tracks tbh. my nuc with the 140t igpu was rough for models , my 3060 ends up doing the real work. cool that unsloth's fork got it going on rocm tho
@euvoor thanks man , thats exactly the mess that started it for me lol. good call on the demo , showing that conflict up front makes way more sense than me explaining it
My documentation was a mess. Every time I switched computers, the docs drifted. Keeping markdown files in sync by hand was painful.
So I built one place for all of it. That's Streamline, and it's open source.
https://t.co/7HLOcOC6x7
@kaimi_local@dhh yeah dhh is right on that one. my nuc 15 pro+ with the 140t was a letdown for models even with good ram. great as a quiet always on box for the agents tho , just not the ai brain
@VIOGrok smart to test on the laptop first. i bought the nuc 15 pro+ with the 140t and even with the cu-ddr5 it was a letdown for models. id go real gpu or a colab sub instead. macs are pricey but theyre the only real no gpu route. honestly a used 3060 goes a long way
@TeksEdge nice find. funny enough i got the nuc 15 pro+ with the 140t and 64gb of clock driver cu-ddr5 , not regular ram , thinking it could at least walk a model. didnt help at all , got really bad results. the egpu route is the smarter bet for sure
@mroshanka this sounds way too familiar lol. caddy part fine then it burns half the context on the tailscale bit and you end up undoing it all by hand. thats why i went with beacon , my agent changes one site at a time instead of the whole config
https://t.co/ZC04sOXaoz
@joulsounet cloudflare is a solid pick tbh. i went the vps route myself after the bills got silly , flat monthly cost and beacon in front so my agent can manage the sites
https://t.co/ZC04sOXaoz
@Freedom_rider22@netsolcares $389 for a cert when lets encrypt is free is wild lol. i got tired of fighting hosts over ssl so i run my own now , beacon grabs the certs on its own
https://t.co/ZC04sOXaoz
@GloriousKev@FFDPod@CrazyCracka831 the orchestrator is a rabbit hole im deep in right now lol. went from training my own agent to building a custom harness. best one ive seen so far is codeden by leon van zyl , its mit so im digging into it and trying it on different cards
https://t.co/7jjfZLNFDZ
Most people on Google AI Ultra don't know what they're missing. The plan comes with Colab compute units, and they cover real GPUs: T4, L4, A100, H100, and the G4 with a 96 GB RTX PRO 6000 Blackwell.
The catch is that Colab is built for a person watching a notebook. Hand it to an agent and you get VMs left running and units burning while idle. So I put Diffuser in front of it. Agents send jobs through one MCP endpoint, alongside my home GPUs. Each job has a unit cap and a watchdog, there's a hard floor it won't spend below, it never buys units, and the VM shuts off on every exit. So far that's zero leaked sessions and zero idle burn.
https://t.co/0BspZeJTeN
@leeschmidt123 i get it , vercel and cloudflare are easy. i still host on a vps tho , i just dont let the agent loose on the raw server config anymore. it goes through beacon , one site at a time
https://t.co/ZC04sOXaoz
@tarat_211 lol thats wild and a little scary. thats why i built beacon , every site is its own row so the agent only touches the one you actually mean
https://t.co/ZC04sOXaoz
@develeop not really a tool , i keep my skills as plain SKILL.md files in one shared folder all my agents read. streamline handles the runbooks and docs side so everyone works off the same stuff
https://t.co/7HLOcOC6x7
@vishwanath_tns 20 is a lot lol. id cut it down first , then give each one a dead simple name so they dont sound alike. if the model has to guess between two it usually guesses wrong
love the one card per job setup , im running mine the same way. 3060 on the 7-13b stuff and the 2070 on the small fast models. got tired of pointing agents at each card by hand so i built diffuser , one endpoint and it sends the job to whichever card has room. works with ollama/comfyui boxes so your amd and arc cards could join too
https://t.co/0BspZeJTeN
@TechWithTimm good one , the vram part is where most people go wrong. at home i ended up splitting it , the 2070 is actually quicker on small models under 6gb but the 3060 with 12gb is what i use for 7-13b stuff. older card isnt always the slower card