Alex Back

[Alex Back @haveisaidthat]

@keennay Imagine the price of a 256gb rtx pro…

[Alex Back @haveisaidthat]

@LLMJunky Why gamer needs it when all games are running on browser now?

[Alex Back @haveisaidthat]

@RinoTheBouncer Cheapest 5090 seems to be zotac one with 32gb ram: 6500, cheaper than the card itself. lol

[Alex Back @haveisaidthat]

@HardwareUnboxed I regretted buying a 5090 for a few weeks because m5 came out as soon I got the 5090 shipment. Then I realized I can run qwen27b at 360 t/s with 5 sessions. Then I realized with context lookup, I am running 700 t/s…

[Alex Back @haveisaidthat]

@IamKyros69 Yes that is where local AI starts

[Alex Back @haveisaidthat]

@imnotchalk I am still looking for a good m5 ultra recipe that gives 150+ for 4-8 concurrency.

[Alex Back @haveisaidthat]

@rohit3a Yes it is based on everyone’s specific project. Mine analyzed all past sessions and set an auto off at 300k

[Alex Back @haveisaidthat]

@wholyv Why do you compact? All my sessions auto close at 300k

[Alex Back @haveisaidthat]

@Youssofal_ I have three machines, running qwen image plus Omni, qwen 27b, and qwen flash. I don’t think it is right but there is no alternative either

[Alex Back @haveisaidthat]

@MiaAI_lab The annoying thing is concurrency. How much do you get running 4x 100k+ context?

[Alex Back @haveisaidthat]

@TheGeorgePu The only feasible one is qwen flash next, on either tensorfold or mlx-serve. Speed is good with 4 to 8 streams. You can feel the heat when you put hands on it. Otherwise it is relatively quiet, but blows out hot air. Glm 5.3 flash hasn’t been optimized yet and is slow slow

[Alex Back @haveisaidthat]

@twid So they do allow ordering more than two. I placed an order just in case.

[Alex Back @haveisaidthat]

@harisahmad59 The frightening thing is: when they published deep search, I prepaid 1K. I am seriously thinking to move from tavily and exa to perplexity search to use up the credit.

[Alex Back @haveisaidthat]

@MiaAI_lab M5 ultra looks so cheap now

[Alex Back @haveisaidthat]

@dennisyu Search 20000mah and 190w on Amazon, $33

[Alex Back @haveisaidthat]

@ashxhart Claude said “Concurrency now works: eight chat streams at once reached 433 tokens/second combined (534 on code), and every reply matched its solo run. The earlier reason for ruling TensorFold out, that it served one user at a time, no longer holds for the 27B.” This is m5 256

[Alex Back @haveisaidthat]

@MiaAI_lab Hmm I was trying to cancel my Google AI subscription. Even Claude couldn’t figure out. Maybe that is a test I should ask argon to do.

[Alex Back @haveisaidthat]

@MiaAI_lab Yeah… if Anthropic offers a 5-year one-time payment for max plan at $200, I will seriously consider buying it.

[Alex Back @haveisaidthat]

@johnternus It is a shame for Apple to sell this without optimizing for AI models, and leave the community to do heavy lifting. Sponsor major developer on mlx-serve, omlx, tensorfold, etc please.

[Alex Back @haveisaidthat]

@ashxhart I think you need to contact Apple to get a grant. It is a shame that Apple doesn’t do this itself.