Morrowmake

Exploring AI. Experiencing the agony of insufficient GPUs.

[Morrowmake @Morrowmake]

I've been working to benchmark the performance of different coding agent harnesses. Somewhat surprisingly, @badlogicgames's Pi takes the lead.

Check it out!

https://t.co/6QxdtImqPv

[Morrowmake @Morrowmake]

@GPTWare Ngl when you said 512gb ram before I assumed you meant ddr4. How many kidneys did you sell to make this happen? Looks awesome.

[Morrowmake @Morrowmake]

@0xSero His point is that running a good model locally is prohibitively expensive because of hardware/power/etc. Unless the local AI community can make powerful and efficient hardware widely available I don’t see what “taking this seriously” means/does?

[Morrowmake @Morrowmake]

@GPTWare Thanks. Trying to decide where to sink my money… The Sparks seem to be good value right now for local inference but as you say given the 273GB/s bandwidth I fear they will age poorly. The Pro 6000s are obviously more capable but just so damn pricey.

[Morrowmake @Morrowmake]

@0xSero What's the performace like?

[Morrowmake @Morrowmake]

@Tech2Wild Curious on your thoughts:
4 x A6000s
vs
2 x Sparks
vs
1 x RTX Pro 6000?

Pricing is somewhat comparable…

[Morrowmake @Morrowmake]

I have been a GB10 hater, but this is cool. Of course everyone would prefer to run it on 4 x RTX Pro 6000s at much higher speeds, but the price difference is ~$30-40k.

[Tech2Wild @Tech2Wild]

What 30 TOK/s of GLM 5.2 on 4xDGX Sparks looks like in Real Time 🔥🔥🔥

[Morrowmake @Morrowmake]

Omp

[Michigan TypeScript @MiTypeScript]

~60% Fable cost cut by transparently turning the code into an image and having the model OCR it. WILD idea. also hilarious. https://t.co/c318SgHqBK

[Morrowmake @Morrowmake]

@0xSero It’s good for sure. Best depends on how high a premium you place on token efficiency.

https://t.co/twQFuV9BVc

[Morrowmake @Morrowmake]

@sudoingX I agree it was an epic fumble, but I don't see why China would flood the consumer market any time soon? There's a reason nobody is targeting consumer hardware right now, data centres / enterprise customers are just more profitable. Also, that kind of scaling will take years.

[Morrowmake @Morrowmake]

@mitsuhiko I can only assume removing the dead weight that is Claude Code can only help.

https://t.co/5K15fGhJ9G

[Morrowmake @Morrowmake]

@onusoz If you could do that, we wouldn’t be spending generational wealth building data centres.

[Morrowmake @Morrowmake]

@47fucb4r8c69323 I don't pretend to know what sellers are thinking - this could all just be a result of flows and leverage rather than anything fundamental.

But in general doesn't this just mean the supply of compute has unexpectedly increased? It's not necessarily skepticism about demand.

[Morrowmake @Morrowmake]

@LottoLabs You may be interested in https://t.co/5K15fGhJ9G

More testing and writeups to come.

[Morrowmake @Morrowmake]

This was always the endgame, and it's just geopolitics.

1. US and Western economies are heavily based on knowledge work and software.

2. AI commoditises and reduces the value of knowledge and code/software.

3. It follows that making AI cheaply and widely available is a Chinese strategic interest, and it is only amplified as more US resources get poured into its frontier labs.

4. China will retain its unique advantage in manufacturing.

It's atoms over bits, geopolitics edition.

[Xiaoyin Qu @quxiaoyin]

I expect @AnthropicAI to increasingly compete against their customers and distill their customers’ data. Chinese models wipe out their margins, leaving no margins on the model layer itself. If Chinese models deliver similar results, Anthropic can’t even compete against US neoclouds in inference pricing as they don’t have enough compute. And it’s getting harder for everyone to pay huge premium even if they stayed frontier. (Most tasks are good enough with cheap models)

In order to justify its 1 trillion valuation, they have to go into applications or services. When they sell both vertical outcomes and models, their enterprise customers would worry and pivot away from Anthropic whenever they can.

OpenAI will be fine with its huge consumer user base where there is no such dynamic.

[Morrowmake @Morrowmake]

@lbrealdeveloper @0xSero @LottoLabs No problem! Was curious to find out for myself. :)

[Morrowmake @Morrowmake]

@0xSero I like that it has a skills tab, it annoys me that codex doesnt

[Morrowmake @Morrowmake]

@0xSero @LottoLabs because its good

https://t.co/5K15fGhJ9G

[Morrowmake @Morrowmake]

@TheAhmadOsman Free VRAM for the plebs?

[Morrowmake @Morrowmake]

@jomdont @petergyang They should raise prices on their subscriptions instead of making models unavailable to subscribers. People subscribe on the understanding they're going to have access to the most current models - price accordingly.