@aschilling 192Gb (24GB) LPDDR5X modules are supposedly available, it's what Gorgon Halo (AI Max 495+) will be using to achieve 192GB of memory.
Given that leaks point to a 1280-bit bus for crescent island, with 64-bit modules, that means 20*24GB=480GB total.
@FelixCLC_@Gok Fair, I was just thinking about models that could fit within a single node (with 4~8 accelerators), where the overhead would be way smaller.
@idoccor@FelixCLC_ Same, I saw the blackhole announcements but hadn't even looked at the prices assuming it'd be out of reach for a regular consumer.
I'm honestly tempted
@Gok@FelixCLC_ May be talking out of ignorance, but for hyperscalers wouldn't such MoE models save a lot on compute & improve concurrency?
Load entire model across many GPU, each request should only got to a specific expert at "expert_param_flop", whereas other experts are free to tackle more?
@FelixCLC_ 100%, if anything it just shows that compilers still have room for improvement.
We should not be glorifying having to do asm by hand, but that is a really powerful knowledge that can help compiler devs to know where to improve as well with a real use scenario.
@kabumcombr tentei fazer uma compra com meu cupom de aniversário, porém tive de fechar o aplicativo no momento. Ao tentar realizar a compra de novo com o cupom, diz que ele se tornou inválido, mesmo sem eu ter usado ele em uma compra, como posso prosseguir?
@CrucialMemory Any spoilers possible on the availability by next week already, or only by the end of the month?
I'm in a weird scenario where my purchasing window will be only during next week haha
yesterday aws launched support for appending to objects to S3. given the limitations, there are some skeptics, but i think it's going to lead to a major shift in cloud architectures. https://t.co/soSJCRzWiv
@SebAaltonen Were your compile tests done on windows or Linux? Asking because I found my M3 Max to be slower than my 5950x on compile tasks, but I've also noticed that Windows is way slower (from 1.2 to 2x) than Linux for such tasks.
Before moving to esphome, it had no deepsleep, comms were over serial telnet, and it didn't have any history tracking. But there's one comment in the code that I'm still really proud of, it just ended up looking so pretty and was actually really useful during this weekend
Finally finished a project of mine that was in the dust drawer for more than I'd like. Esp8266 with a cute 16340 battery for some temp+humidity+light measurements using esphome + home assistant (previously it was just some basic arduino code)