Inference Scaling and the Log-x Chart:
2024 saw a switch in focus from scaling up the compute used to train frontier AI models to scaling up the compute used to run them.
How well is this inference scaling going?
1/
The Scaling Paradox:
AI capabilities have improved remarkably quickly, fuelled by the explosive scale-up of resources to train the leading models. But the scaling laws that inspired this rush actually show very poor returns to scale. What’s going on?
1/
https://t.co/gm32S8SpDX
Ilya Sutskever had a conversation last year with Jensen Huang but still, that made me rethink the idea of 'Predicting the Next Word'
Listen to what Ilya said on this.