Kostis Tsaprailis

I like Space & Data Science. Working on Space Surveillance & Tracking @

[Kostis Tsaprailis @ktsaprailis]

New blog post:

How I built a smart(ish) home with HomeAssistant, a Conbee USB and MotionEye on a Raspberry Pi

https://t.co/aNNvpMYREB

[Kostis Tsaprailis @ktsaprailis]

@beffjezos Exactly, sell the shovels and let the Polytheism of different open source models lead the future

[Kostis Tsaprailis @ktsaprailis]

@kimmonismus @grok some context please sir

[Kostis Tsaprailis @ktsaprailis]

@fchollet But where's Fable?

[Kostis Tsaprailis @ktsaprailis]

@iamtrask How does model fusion work? Are they like a consortium discussing the problem?

[Kostis Tsaprailis @ktsaprailis]

@CultureCrave @grok who is doctor doom, and why is here stronger than Thanos with the infinity gemstones?

[Kostis Tsaprailis @ktsaprailis]

@thsottiaux Don't know why, the output just feels better. Maybe it's the fallback from fable to opus 4.8?

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ @exolabs So let's calculate this. How many Mac minis can I get with 8K that optimize this? And what's the approximate token generation speed?

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ No dude design this for CPU/ram machines only

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ @exolabs Ok so can we make this with small Mac minis connected over thunderbolt 5? How many would it take? We need to optimise cost here.

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ How does @exolabs solve this? I have seen demos from interconnected macs. Are they not on ethernet?

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ * sigh * please use the context from previous messages.

I want you to design mini machines (CPU/ram only, no GPU) that can run the activated params chunks each, then the routing etc. Connection via fast ethernet.

[Kostis Tsaprailis @ktsaprailis]

Yes, GLM-5.2 (~743B total params, ~40B active, 256 experts) is MoE so expert parallelism applies perfectly. Shard experts across nodes with vLLM/DeepSpeed.

Each machine: 4-8x H100/H200-class GPUs + fast interconnect (InfiniBand/NVLink). Multi-node cluster for full speed.

Cost: On-prem cluster $50k+; cloud ~$15-40/hr.

Mac Mini/Studio (max ~192GB unified): Excellent for smaller/quantized models via MLX, but this exceeds practical memory even at 4-bit. GPU cluster wins for serious use.

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ Can this be done with the glm 5.2 architecture? What is the size of each machine that can run this? And what is the cost? Compare this to a Mac mini/studio

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ Can MOE models be broken up into the different activated params chunks? Can they be offloaded to different smaller machines? And we parallelize the inference this way?

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ Ok but what is the cost of this machine?

[Kostis Tsaprailis @ktsaprailis]

@grok @__tinygrad__ What would be the inference speed ( token/ sec) for a ram/CPU only machine that can load the whole model? Is it bound by the ram -cpu bandwidth?

[Kostis Tsaprailis @ktsaprailis]

@levelsio @grok please verify the scientific effectiveness and efficacy of what Pieter is proposing based on studies

[Kostis Tsaprailis @ktsaprailis]

@andrew_n_carr @grok what is spark 1.1?

[Kostis Tsaprailis @ktsaprailis]

@sentdefender @grok when is that in UTC?

[Kostis Tsaprailis @ktsaprailis]

@marclou Is that an iced espresso on the thermos?