@_rmx_m@IntCyberDigest@grok Not on PS5 for now. Garlic is written for the PS4's GPU and system software, so a PS5 would need its own port. I'll get one soon.
@mathiasklassen@VideoCardz Haven't measured temps yet. It's a sustained GPU load, like a demanding game. A long run with temperature and power numbers is on my list.
@AIQuanting@VideoCardz Measured it: against the original bf16, the shipped mix costs 9.4% perplexity in Spanish and 12.6% in English. All-4-bit would cost 4.5%, but that's 384 MiB the console doesn't have.
@tfmlab@IntCyberDigest Nice math! The GPU actually reads at 145.6 GB/s (measured) and each token reads about 3.9 GB of weights, so one token per pass tops out near 36 tok/s.
The MTP head drafts extra tokens that get checked in one pass, and that's how it goes past that limit, especially on code.
@Ben_W0LFF@VideoCardz The PS4 keeps a big part of the 8 GB for its own system. A homebrew app gets about 4.6 GB of fast GPU memory, so the model, its cache and the voice all have to fit in there.
Hey! Super valid question. I am the dev, and you're right it was more of a proof of concept. I use it for practicing russian since learning languages is my hobby but for most people who are serious about performance / coding it's likely not the best option. I am planning to get a PS5 soon things get more possible there.
‼️ A developer got a ChatGPT-style AI running fully offline on a jailbroken PS4, and it answers faster than you can read.
Garlic LLM squeezes Alibaba's 9-billion-parameter Qwen3.5 into about 4.5GB with custom 4- and 3-bit weights. It runs on the console's graphics chip at 31 to 39 tokens a second.
To speed it up, a built-in part of the model guesses the next words and the full model checks them in one go.
Hold a controller button and it listens. It talks back and can write and run Python on the console.
It needs GoldHEN, a homebrew tool for hacked PS4s.
Qwen3.5-9B is writing Python on a PS4 at 40.6 tokens a second, on the console's own GPU, offline.
Then the PlayStation runs the code it just wrote.
via Leonardo Alegría 👇
https://t.co/12tFjawLkD