Alex Brandes ²

CTO @conste11ation | generating utility

[Alex Brandes ² @codebrandes]

Join us. True 64GB Ampere class NVIDIA cards. PCIE 2.0 x4 link sounds limiting but we’re getting great speeds with pipeline parallelism anyways. 6,000 tok/s prefill and 200+ tok/s decode is pretty nice.
https://t.co/po7Wu7XcF3

Or you can mod them to x16 and do tensor parallelism. Suggest checking out @Morrowmake’s work on that.

[Alex Brandes ² @codebrandes]

@Morrowmake You’re a bit ahead of me but I’m still throwing some compute at EXL3. Basically at half the prefill speed of W4A16, like you said. I’m hoping there’s more juice to squeeze though still.

[Alex Brandes ² @codebrandes]

@ItsmeAjayKV New Qwen models for the people 🙌

[Alex Brandes ² @codebrandes]

Disclosed vulnerabilities are already starting to slope towards infinity.

Not really the singularity I had hoped for, to be honest.

[calle @callebtc]

I'm tired boss

[Alex Brandes ² @codebrandes]

@PeasantSmith Not sure what we’re talking about but these speeds are unacceptable. Back to the lab until we get over 40 tok/s.

You got this 🤘

[Alex Brandes ² @codebrandes]

@Rundychops @BenJorgensen @Conste11ation @PacaSwap Did you fill out the form on https://t.co/3aOOm0uGMQ? If so and you haven’t heard back, send me a DM and I’ll make sure we get it sorted out for you.

[Alex Brandes ² @codebrandes]

@kamilskowron I wonder if you could use some of the tricks that cmpunlocker uses on CMP 170hx mining cards to achieve PCIE 2.0. You might even do ok with pipeline parallelism on 1.0 since you’d be doing just a few layers per card.

https://t.co/JixvGXCn9F

[Alex Brandes ² @codebrandes]

Yo, WTF?

[solst/ICE of Astarte @IceSolst]

I don’t understand? How

[DeepSeek @deepseek_ai]

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.

🔗 https://t.co/JGAik1kgk0

1/4

[Alex Brandes ² @codebrandes]

@Morrowmake That makes sense. I was playing around with developing some EXL3 kernels last week focused on 27b initially but the implementation is still pretty rough. Maybe I'll just adopt yours and try to add optimizations. I'll make a PR if I find anything worth sharing.

[Alex Brandes ² @codebrandes]

@seanphan Let me know if you’d be open to a PR on your repo. I believe this is the fastest public recipe for Qwen Flash Next on 2x 170hxs.

[Alex Brandes ² @codebrandes]

New recipe from the home lab on 2x CMP 170HXs 👨‍🍳

Qwen 3.8 Flash Next

Single stream: up to 226 tok/s
8 concurrent: up to 840 tok/s
Prefill: 6000+ tok/s

Running W4A16 quant - PP2 - PCIe Gen2 x4 links

These cards are putting in work.

https://t.co/VX7TU9PKJC

[Alex Brandes ² @codebrandes]

@joaopfc 🏒🚀

[Alex Brandes ² @codebrandes]

@aye_pete @ItsmeAjayKV We have over 300 paid models available also! Free models rotate every few weeks to give everyone a chance to try them out.

[Alex Brandes ² @codebrandes]

The state of GPU market right now.

[RTVisual @RT_Visual_on_X]

Chinese bride gifts her partner a flagship GPU at wedding — he breaks down in tears

She handed him an ASUS ROG Astral RTX 5090 that costs around $12,000

[Alex Brandes ² @codebrandes]

@Merkaloid @ItsmeAjayKV Yep, free models have limited usage based on capacity. Throughput is the same as the paid models though.

[Alex Brandes ² @codebrandes]

@kbw2f6595r @ItsmeAjayKV We do not sell any user data and only route to providers with no-training agreements.

[Alex Brandes ² @codebrandes]

@karankendre GLM 5.3 Flash and Qwen Flash Next are also free on @_GateAI right now at 1M context with no data training.

I feel like a spam bot today but please use our free models!

https://t.co/Y8Kmdiz1xp

[Alex Brandes ² @codebrandes]

@Smlay_ero GLM 5.3 Flash and Qwen Flash Next are also available for free on Constellation Gate at 1M context. You can use any harness you want.

We also don’t train on your data!

https://t.co/Y8Kmdiz1xp