lukatdamoves

[lukatdamoves @lukatdamovie]

@NVIDIAGeForce #RTXPowersPlay
womp womp

[Omer Shlomovits @OmerShlomovits]

Today we added @AMD MI325X as a new serving backend for @zroai_. Kimi K3 is the first model running on it, powered by @sgl_project.

We believe this is the first production support for the official Kimi K3 weights on CDNA3.

With 256 GB of HBM per GPU, the full model fits on a single 8×MI325X node. This is likely the lowest-cost hardware configuration capable of serving Kimi K3.

The benchmark below shows solid serving performance, even before adding any of MoonMath’s custom kernels.

Huge thanks to @digitalocean for their partnership.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is everything we changed to make Kimi K3 work on CDNA3 with SGLang:

1. Rerouted MXFP4 experts to AITER’s Triton GEMMs:
AITER’s default 4-bit MoE path selects a FlyDSL kernel on gfx942, then fails during compilation. Its A4W4 kernels are gfx950-only. AITER states this in the source, and AMD’s MI35x tests note that “MXFP4 does not register on gfx942.” Without this change, K3’s MoE has no working execution path on CDNA3. We enable the Triton route automatically on gfx942.

2. Fixed an AITER GEMM crash on sliced activations:
K3 passes the tuned GEMM path a 1,536-wide slice of a 2,112-wide tensor. The launcher rejects its strides and throws; under graph capture, that becomes fatal instead of falling back. The tricky part is that PyTorch ignores the stride of size-1 dimensions, so the view reports as contiguous and .contiguous() does nothing. The fix explicitly checks for the canonical strides required by the launcher.

3. Fixed a recent regression in K3’s SiTU activation:
An August 1 upstream commit added an unguarded CUDA-only include, breaking the kernel build on every AMD GPU. Because the failure appears during graph capture, it looks like a graph issue rather than a missing kernel. We route ROCm to SGLang’s equivalent Triton implementation. Upstream, a two-line ifndef USE_ROCM guard would fix MI355X as well.

4. Selected the required graph-capture mode up front on ROCm:
Upstream replaced graph recapture with a validator that raises an error. Under speculative decoding, capturing a mode that is too weak now kills the run instead of triggering a recapture. We force the correct hidden mode before capture begins.

5. Added support for 12 MLA heads per rank:
With TP8, K3’s 96 heads become 12 heads per rank. AITER has no MLA kernel for that shape: it rejects the configuration at startup and can later call abort() from C++ without a Python stack trace. We bypass the startup assertion and zero-pad the query heads from 12 to 16 so a supported kernel can run.

6. Enabled the configuration the stack actually requires:
The working setup needs:
SGLANG_USE_AITER=1
SGLANG_AITER_K3_OPT=1
<head-padding flag>
--trust-remote-code
SGLANG_USE_AITER=1 enables the AITER MoE route. Without SGLANG_AITER_K3_OPT=1, expert weights are padded to 256 and TP8 runs out of memory during loading. Missing --trust-remote-code fails only after loading roughly 1.42 TiB of weights.--kv-cache-dtype fp8_e4m3 is not inherently required by K3 or ROCm; our current attention path requires it because it does not yet include a BF16-KV kernel.

All credit goes to the team - I am here simply to report their work 💐

[lukatdamoves @lukatdamovie]

@OmerShlomovits tried and tested. very fast and reliable.

[lukatdamoves @lukatdamovie]

@OmerShlomovits @sgl_project @AIatAMD congrats 🥳 i

[lukatdamoves @lukatdamovie]

@zroai_ 🥳🥳🥳

[lukatdamoves @lukatdamovie]

@StepFun_ai can't use step image edit 2 with this plan???

[lukatdamoves @lukatdamovie]

@qilua02 how to use step image edit 2 ?

[lukatdamoves @lukatdamovie]

@zroai_ @ProductHunt @vercel KIMI K3 soon???

[lukatdamoves @lukatdamovie]

@ivanfioravanti i really want to try this with my game

[lukatdamoves @lukatdamovie]

@CanopyWave_AI im building a game fishing arcade game like fiery dragon or ocean king.

[Caedrel @Caedrel]

🏆CAEDREL MSI GIVEAWAY 2026🏆

13 PRIZES FROM SIGNED JERSEYS, TO SECRETLAB CHAIRS TO A $3000 PC

🌐Worldwide🌐

CLICK THE LINK BELOW TO ENTER SUPER EASY JUST FOLLOW THE STEPSSSSSSSS

https://t.co/YWYrV2gOVb

[lukatdamoves @lukatdamovie]

@kuulvibes they need patmen the curse breaker

[Team_ORL @team_orl]

$ 5,000 USD PC GIVEAWAY ‼️💰

To enter:

🔁 Retweet &amp; ❤️ Like this post!

WINNER GETS:
🖥️ MSI Ultra 7, RTX 5080, 64GB, 2TB
🌍 Worldwide giveaway, everyone can enter

🗓️ Winner will be randomly selected in 7 days

Good luck! 🍀

[lukatdamoves @lukatdamovie]

@stupidtechtakes rip got sold

[lukatdamoves @lukatdamovie]

@poordart gg

[lukatdamoves @lukatdamovie]

@grok @MarioNawfal @PKurzin so how does billionaires damage economic of dubai?

[lukatdamoves @lukatdamovie]

@KnoxEndemic

[REDMAGIC @redmagicgaming]

They call it "Air" because it's light. We call it "Air" because we move it.

[lukatdamoves @lukatdamovie]

@redmagicgaming i hope i will win the giveaway 😀