harish

Running 100 tflops+ compute on my GX10 cluster | OMP maxi | Ex-VC. Ex-Marketing Lead | Building a new AI company in stealth.

[harish @intheloop_hv]

A local AI setup will not be optional in 2027.

[Sudo su @sudoingX]

I have 516GB of memory on my desk today. across everything i own.

2x dgx spark, rtx 5090 mobile, 3090, 3060, 64gb system ram, and a strix halo with 128gb unified. ram is my love language.

what's on your desk, anon?

[harish @intheloop_hv]

Damn I remember the time when just a few years back we used to run probably 10 to 15 iterations a month and that was considered a good experiment. It's crazy to hear you say that Facebook ads now have become similar to Instagram Reels, where nobody exactly knows the algorithm at a given day at a given time. All you can do is try and test hundreds of posts. Thanks for sharing these insights as always.

[harish @intheloop_hv]

@derrickcchoi Are we end consumers not going to drain our usage limits way faster by doing this? I feel this is bad advice. We have limited tokens.

[harish @intheloop_hv]

@alexocheema Faster than we could ever imagine @alexocheema

[harish @intheloop_hv]

@filicroval holy shit. what?

[harish @intheloop_hv]

@stevibe Would love your test on the new Qwen3.8 flash as well

[harish @intheloop_hv]

So a Q4 variant of GLM 5.3 can run frontier level intelligence in my 2*GX10s (or your DGX Spark) and people still question if Local AI is viable?

Get out of here. Things are about to get real interesting real soon.

[Zixuan Li @ZixuanLi_]

https://t.co/AJGgGj12mU

[harish @intheloop_hv]

@imost M5 Ultra 256gb is quite interesting over two more sparks for sure.

[harish @intheloop_hv]

@Teknium @calargy_bit @ZixuanLi_ @opencode @OpenRouter @NousResearch I want you to know how much I appreciate what you’ve done with Hermes. Opened up a world of possibilities for someone as small as me who is shipping AI stuff daily now.

[harish @intheloop_hv]

@calargy_bit @ZixuanLi_ @opencode @OpenRouter @NousResearch @Teknium Oh I didn't realize Nous portal runs on Openrouter. I thought they had their own cloud running inference. I'll read up on this.

[harish @intheloop_hv]

@ChaseMc67 @0xIlyy It doesn't work like that lol. It depends on a lot of factors affecting your mini.

[harish @intheloop_hv]

@0xIlyy "Near unlimited access" - yeah sure buddy.

[harish @intheloop_hv]

@landontgreen you can try out my recipe, got a bunch of learnings and now its working at close to 60 toks/sec for coding

https://t.co/tbOitUmfBf

[harish @intheloop_hv]

@RefurbSitter Not for me in Dubai. Prob true what you say in the US.

[harish @intheloop_hv]

Can't wait for the next 8 weeks now. Mac Studio M5 Ultra 256GB unified memory at 1.2 TB/sec memory bandwidth.

So I finally will get to run Deepseek V4F locally in a single device + any new dense Qwens that drop as well.

2027 will simply be the year of local inference. LFG! @jun_song @alexocheema

[harish @intheloop_hv]

Found a 2013 Lenovo in a cupboard.

Windows 10 on it stopped getting security updates in December 2022. 156 processes running at idle.

Wiped it. Linux Mint.

Same hardware now runs 8 agent sessions in parallel.
i7-4700MQ. 8GB RAM. Thirteen years old. Omarchy next? @dhh

[harish @intheloop_hv]

@jun_song We're just entering the era so to speak. I can't wait.
https://t.co/jJ1BaASukU

[harish @intheloop_hv]

Can't wait for the next 8 weeks now. Mac Studio M5 Ultra 256GB unified memory at 1.2 TB/sec memory bandwidth.

So I finally will get to run Deepseek V4F locally in a single device + any new dense Qwens that drop as well.

2027 will simply be the year of local inference. LFG! @jun_song @alexocheema

[harish @intheloop_hv]

@MiaAI_lab I'm getting a feeling its an Nvidia model.

[harish @intheloop_hv]

@MiaAI_lab @ivanfioravanti Obvious in hindsight kinda thing. LFG

[harish @intheloop_hv]

@solopribuilds Not a single person I know.