@maxbittker It would be very interesting to make them follow the ironman guide linked below + allow them to create scripts that manipulate the mouse for harder dynamics like blackjacking
https://t.co/McjwUxNRok
It's very cool that Apple shipped a 20B parameter on-device.
You can't put 20B parameters in RAM at any reasonable precision. To make it work they are using pretty exotic architecture by today's standards.
A small model predicts from the query (or prompt) which experts to load from Nand into RAM. The key distinction from a typical MoE is that you do this once per query and then generate all the tokens with the same experts (instead of switching the experts for every token).
The best advice I always give to newly exited founders:
Open a Vanguard account.
Not Fidelity. Not Schwab.
Vanguard.
'Smart advice,' You might think. 'They do have the lowest fees..'
Wrong.
Their interface is so awful, you will never trade..
Has made my clients millions.