Brian

Creator of the MXFP4 vLLM build for Radeon AI PRO R9700. Fastest 4-bit LLM inference I've measured on RDNA4.

[Brian @bkuyper]

Inference engine built in rust currently targeting AMD R9700 GPU's.

@AIatAMD #AMD #R9700 #LocalAI #LocalLLM

https://t.co/QlJ8MJOpMJ

[Brian @bkuyper]

@1337hero @techforgooduse I don't know what you are testing but your test is f'ed up. Nobody is getting almost 50% of the speed on decode as what they hit on prefill. You realize I run the discord server (Launch80) that is the primary development of Radiance. I am well familiar with the speeds

[Brian @bkuyper]

@1337hero @techforgooduse There is no way you are hitting 1150.0 tok/s decode . lol, that is past the peak of the hardware even

[Brian @bkuyper]

@loktar00 You should of went with R9700's

[Brian @bkuyper]

@1337hero all the teasers are killing me

[Brian @bkuyper]

@_jarva_ @AIatAMD I didn't post QFN results as on QFN it matters more even what hardware you have. This isn't some sort of comptetition so I don't like posting results like this. But here are the results on my hardware so you can compare.

This is powercapped at 210w

[Brian @bkuyper]

Inference engine built in rust currently targeting AMD R9700 GPU's.

@AIatAMD #AMD #R9700 #LocalAI #LocalLLM

https://t.co/QlJ8MJOpMJ

[Brian @bkuyper]

@techforgooduse @AIatAMD it should work with 1 or 4 also

[Brian @bkuyper]

@cheeez42 @Beelinkofficial Yeah I have 2 of them and they have been solid

[Brian @bkuyper]

@1337hero Now you gotta try https://t.co/U3TzgAgSSR

[Brian @bkuyper]

@AIatAMD

[Brian @bkuyper]

@Furion458 @AIatAMD yes

[Brian @bkuyper]

@AIatAMD

[Brian @bkuyper]

@AIatAMD

[Brian @bkuyper]

@1337hero 210 powercap and -75 undervolt works good

[Brian @bkuyper]

@Da7_Tech I sure hope so

[Brian @bkuyper]

@0xdominus @grok where is this image from? is it real?

[Brian @bkuyper]

@cheeez42 Also, how about all the warehouses literally sitting full of GPUs that they aren't even using

[Brian @bkuyper]

Super excited to see what this brings us and glad to see some more competition in these model ranges.

#LocalAI #GLM55 #qwen4

[Brian @bkuyper]

@AestherML It needs to be replaced at this point.

[Brian @bkuyper]

@1337hero @TeksEdge it's so obvious at this point it's a joke as soon as Kimi K3 dropped a few days ago I figured artificial analysis was going to update the scoring. They have done this every time now

[Brian @bkuyper]

@1337hero Tim Cook was just an executor which I guess he did an okay job at that. But from an innovation standpoint it's been terrible