what if instead of using nvlink, we trained a separate model to predict the gradients instead? maybe everyone at $NVDA isnt as bright as i thought they were
There's an estimated 120,000,000,000,000,000,000+ bytes of DRAM in service today (120+ exabytes).
That's over 14GB per human.
Yet here we are in a RAM shortage.