Builtbyjj

[Builtbyjj @BuiltbyJJ]

@Hi_MINISFORUM estimated pricing and release?

[Builtbyjj @BuiltbyJJ]

@MatthewBerman everytime i think ive put i ding in my usage i take a look and it be still at 98%. im like how?!lol

[Builtbyjj @BuiltbyJJ]

Spent my Sunday building my threadripper pro with 2 - rtx 4500 pro running 24.04 Ubuntu LTS server. Continuing today with connecting my spark through a qsfp 100gbe. My plans are for spark to handle prefill and rtx to handle decode.

[Builtbyjj @BuiltbyJJ]

@abacaj The real ones are fine tuning models. Trying to run frontier models at the time is slow. But it shows you that it’s becoming possible and overtime it will continue to advance. Owning your own compute has its benefits.

[Builtbyjj @BuiltbyJJ]

@TheAhmadOsman @guohao_li I need ddr4 RDIMM 8 channel 256gb 3200mhz . Whose got the plug

[Builtbyjj @BuiltbyJJ]

@AlexFinn @SinnerOnPC Go 4500 pro Blackwell instead.

[Builtbyjj @BuiltbyJJ]

@mr_r0b0t I caught a crazy deal at Best Buy on 5060ti 16gb.

[Builtbyjj @BuiltbyJJ]

Some weekend upgrades. Just added another 4500 pro. Next is going threadripper.

[Builtbyjj @BuiltbyJJ]

@peladodelatv AI

[Builtbyjj @BuiltbyJJ]

need help deciding. I'm trying to not having to go threadripper yet. I current have a msi x670e carbon x8 x8 x4 with a 4500 pro blackwell.

This week i am going to pull the trigger on either a

another 4500 pro or 5090 FE (2 slots other 5090s wont all fit unless i water cool)

Do i keep things in the same family and get the 4500 or get a the 5090 for more bandwidth and cuda but now have 2 different type of cards? I do like to mess around with wan2.2 or ltx and would like the faster renders but not a deal breaker.

2: sell my current 4500 pro, keep the cash on the 5090 fe ($4k) put that towards a 6000 pro. ( i have a seller at $8k)

mind you I have a dgx spark to which i will run a disaggregated inference. Spark does prefill and pc handles decoding.

or

3. buy another spark and cluster

[Builtbyjj @BuiltbyJJ]

Started testing disaggregated inference between NVIDIA DGX Spark and an RTX PRO 4500.

Spark ↔️ RTX PRO 4500 P/D benchmark:

Model: Qwen3-0.6B
Stack: vLLM 0.21 + NIXL + UCX
Network: ConnectX-5 100GbE
KV buffer: cuda
Prompts: fresh unique prompts

Prefill → decode results:

run1: 188 tokens / 0.697s = 269.8 tok/s
run2: 189 tokens / 0.586s = 322.5 tok/s
run3: 192 tokens / 0.627s = 306.3 tok/s

avg: 299.5 tok/s
median: 306.3 tok/s
range: 269.8–322.5 tok/s

More importantly, metrics confirmed external KV transfer:

- 38.5MB transferred over NIXL
- 0 failed transfers
- prompt tokens came from external_kv_transfer

This was a tiny model, so the benchmark itself is not the point.

Spark performed prefill, the RTX PRO 4500 handled decode, and KV actually moved across the wire.

I just got another 4500 pro so im going to test larger models. just wanted to get it work. i was super excited! This is my first time doing something like this and im still learning.

[Builtbyjj @BuiltbyJJ]

@TheAhmadOsman Cloudflare the way to
Go. Been doing this as well.

[Builtbyjj @BuiltbyJJ]

codex is down.

[Builtbyjj @BuiltbyJJ]

yeah i did check the status page and it said the same. but i did a search in X and saw many other with "stream disconnected before completion: An error occurred while processing your request. You can retry your request, or contact us through our help center at https://t.co/MysZ2ghYzF if the error persists.

[Builtbyjj @BuiltbyJJ]

@david_nix you did it!

[Builtbyjj @BuiltbyJJ]

@david_nix You’ll be back for it !! Trust !!😂

[Builtbyjj @BuiltbyJJ]

@BassPattern @shiri_shh @grok Yes but the difference is in how it runs in concurrent tasks . This is where it outperforms and prefill as well .

[Builtbyjj @BuiltbyJJ]

I feel this 100% all the side quest and rabbit holes I went through all my younger years and even now.

But they all aligned to my career. A way I think about it is in reference of the movie “slum dog millionaire” where the main character his whole life went through multiple experiences and during that time it didn’t equal to anything . But when he went on the “who wants to be a millionaire” show. He was able to answer all questions as they pertain to a moment in his life. To which all thought he could have been cheating. He goes on to win the show but it was all those side quest that came to play at the end. To which everyone moment of his life aligned.

Dope post