@zephyr_z9@HyperTechInvest Ok, my bad seems I missed it your previous comment was primarily on HBM. Offload not only to NAND, but to dram pool via cxl as well which I have seen very large production deployments
@zephyr_z9@HyperTechInvest Do you know how much memory is used for weights versus kv cache currently? Why does it even matter when parameters are not scaling in the future?
@IvanaSpear Google’s engineer did a presentation about HBF in FMS, so Google must already be in the game. You did great job on optical companies, but you might want to do more research on storage companies before commenting