the tiny corp

We make tinygrad; sell tinybox for the GPU middle class.
Our mission is to commoditize the petaflop.

[the tiny corp @__tinygrad__]

Meet the new tinybox. Configurable. Starting at $7,000.

[the tiny corp @__tinygrad__]

@tkanarsky We're working on all these tricks, it should be able to get even faster. MoE gives you tons of control without much of an accuracy hit.

[the tiny corp @__tinygrad__]

This is the topology of the tinybox red kimi, frontier models at home at usable speeds for a good price. tinygrad has a driver to directly control the NVMe drives from the AMD GPUs.

[the tiny corp @__tinygrad__]

@EricParker It's the most you can get on a single CPU, half the lanes for NVMe, and half the lanes for GPUs to read the NVMes.

[the tiny corp @__tinygrad__]

@marksaroufim For sale on https://t.co/ryGYRcEHIN

[the tiny corp @__tinygrad__]

Run Kimi K3 at home. 40 tok/s interactive. $30k.

[the tiny corp @__tinygrad__]

@secemp9 The website should answer all your questions. https://t.co/ryGYRcEHIN

[the tiny corp @__tinygrad__]

@stake_jevens It's for anyone who wants to buy it

[the tiny corp @__tinygrad__]

@Smallzero @keennay I mean, we'd have to improve the software to achieve the potential of the hardware, it's not like a 1 line test. But if you buy the machine and tell Opus 5.5 to make it fast...

[the tiny corp @__tinygrad__]

@tkanarsky This. There's a lot of expert overlap between subsequent tokens, which is why we need the 128GB of GPU cache. We first had it at 64GB with 9070XT cards, but it was missing too much.

[the tiny corp @__tinygrad__]

@AlpinDale About 56 lbs, I added dynamic weights to the website.

[the tiny corp @__tinygrad__]

@GPTWare hehe, it's full bandwidth :) https://t.co/pBCxaYGJVM

[the tiny corp @__tinygrad__]

This is the topology of the tinybox red kimi, frontier models at home at usable speeds for a good price. tinygrad has a driver to directly control the NVMe drives from the AMD GPUs.

[the tiny corp @__tinygrad__]

@FundedXX Do you have a cheaper option? Or do you have a list of parts from an AI that won't actually work together? How exactly are you connecting to all 128 PCIe lanes of an EPYC?

[the tiny corp @__tinygrad__]

@snapolino I mean, maybe if I was literally hammering a single row. But I have never heard of drives wearing out from reading them normally too much, and they are well cooled in the tinybox.

[the tiny corp @__tinygrad__]

@AlpinDale We don't by default, but assuming there's no export controls blocking us we are happy to. Hong Kong is available in our self checkout.

[the tiny corp @__tinygrad__]

@kimmonismus you see a competitor, you copy the best things from them. if the labs don't want their models distilled, they shouldn't release them. US law says AI outputs can't be copyrighted.

[the tiny corp @__tinygrad__]

@kimmonismus lol "reputation damage" I like that they distill they should distill Opus 5.5 harder to make the best Kimi they can, Astra makes the reasoning into pidgin english which you can see in Kimi K3.1, a little too much RL doesn't have to be distillation.

[the tiny corp @__tinygrad__]

@snapolino I don't think NVMe's degrade from reads, just writes. And yea, the CPU has 4 IOHUBs with 2 x16 PCIe, so a GPU and 4 drives on each.

[the tiny corp @__tinygrad__]

@keennay Haven't tested it, we've been focused on decode. But prefill should be even easier, you can prefetch layer by layer so disk isn't too much bottleneck. At ~100B active params and 1.5 PFLOPS of compute I could see 5000 being doable. This is the cutting edge of tinygrad.

[the tiny corp @__tinygrad__]

@anatolykim8 Haven't tested it, but probably around 80 tok/s and 150 tok/s respectively. This is on the cutting edge of what tinygrad can do, we have an NVMe driver that's directly controlling the drives from the GPUs.