Jason

[Jason @original_el8]

@GarnerFox @TheAhmadOsman @Sentdex Love it!! Enjoy, reach out if there's anything you need help with. There's a bunch of us station owners hanging out in Local Inference Lab's discord if you'd like to join:
https://t.co/PpJewdqxp0

[Jason @original_el8]

@GarnerFox @TheAhmadOsman @Sentdex Love it! Did you see my sidecar recipe?

[Jason @original_el8]

@TheAhmadOsman @Sentdex oh yeah, it's incredible on the station with the 6k sidecar.

[Jason @original_el8]

@TurrinaMartino @itsjustmarky @minyongchoe4 https://t.co/PpJewdr5ey

[Jason @original_el8]

@exploit_intel @MiaAI_lab We’re still working on it. We definitely have more in the tank.

[Jason @original_el8]

@SJRonanMD It does but it’s not awesome I’ve found.

[Jason @original_el8]

@u1tra_instinct @chirag Not exactly correct. It's not just 14.7GB of disk space, it's 14.7GB of VRAM.

[Jason @original_el8]

@therealazzurro @MiaAI_lab Strata doesn’t do what you expect it does

[Markets & Mayhem @Mayhem4Markets]

When a new model drops, I look towards @YourLocalAILab to get it going.

Not only is the website easy to use, but the model weights and docker are both tested to ensure they work.

It's made my local AI experience significantly better. Cutting setup time from hours to minutes. 👇

[Local Inference Lab 🧪 @YourLocalAILab]

Ready to supercharge your local AI hardware? 👀

Build an optimized docker for your @NVIDIAAI RTX 6000 Pro or DGX Spark at https://t.co/xHLxqQ5sjr

[Jason @original_el8]

@SJRonanMD @Sentdex I actually have some kernel updates that I’m working on upstreaming to improve these. Glad it’s working well!

[Jason @original_el8]

@SJRonanMD @deepseek_ai Great job!

[Jason @original_el8]

@SJRonanMD @NVIDIAAI @deepseek_ai @Exxactcorp Love it. Good work. 👍

[Markets & Mayhem @Mayhem4Markets]

Really impressed with @YourLocalAILab's lossless context compression (CSF).

I just gained about 30% more tokens in my KV cache on DeepSeek V4.1 Flash.

The only thing I had to do was download the new weights and update my docker.

I love these guys. Such great work.

https://t.co/QtHTuAT5hp

[Jack Zampolin @jackzampolin]

Updated GLM 5.3 Flash serving today on my production machines. Thanks for the new image @YourLocalAILab! Also this bench sharing tool is amazing

https://t.co/1bJyN8kGrH

[Jason @original_el8]

@keennay @alexocheema @ivanfioravanti @Prince_Canuma @YourLocalAILab We’re very much supporters of all local AI, but we do focus on Nvidia Blackwell in 5000 series, RTX Pro, DGX Spark and DGX Station.

Partly that’s because that’s what our developers have but also because it allows us to focus on cutting-edge features and capabilities.

[Jason @original_el8]

@volatilemarkts @Lcaplmn I think the problem is inference is basically non-competitive on Apple in general even for decode. Prefill is obviously an architectural limitation but decode even doesn't match expectations with their unified memory bandwidth. Kimi on Studios would be mega bad.

[Jason @original_el8]

@BerrosJeremy @MiaAI_lab sort of just a bad use of two sparks.

[Jason @original_el8]

@MiaAI_lab It's slower than two sparks by itself. 10Gbps isn't gonna get anything done here as far as disaggregated inference.

[Jason @original_el8]

@volatilemarkts @Lcaplmn I think what he's saying is that both prefill and decode are faster with just the Sparks than having the Mac in the loop at all. It's just slowing it all down.