Because small models are just automation, which the world already has. Everything is already automated. The primary holdouts are monopolies resisting competition.
The goal is to replace what humans do well and is hard to automate. Humans can react quickly to very minimal information. We respond well to very high variance in our environment. Automation hates variance, because it uses statistics to estimate it, which is very expensive and requires a lot of data.
The dirty secret of AI is it’s just applied statistics. All of it. There is no intelligence at all, it’s pure marketing (look up the origin of the term itself). That’s why the models are so large and require all of the data on earth. The AI start ups are trying to solve high variance with lots of data.
What everyone will painfully learn is that statistical methods don’t scale gracefully, and we are already seeing that even the entire corpus of human knowledge isn’t enough data to guarantee accuracy in all cases, so they use agents (loops) to guess until it looks close enough.
Take software as an example. To use statistics to make a program that can write absolutely any possible software can only be guaranteed by writing all possible software and feeding it into the model. Since that is not practical, we train it on maybe 70% of all software, give it some random number generators for the last 30%, and ask it to write the code 1000x for one program and take the best one.
This sort of works well for boilerplate apps we have lots of examples of (ie programs that already exist), and really terribly for novel or bespoke software (like a new physics model).
It’s all sleight of hand, hiding the cost of creating something by calling it “training” and then showing people how quickly it can regurgitate it later. The only way this is economical is if the valuable artefacts are stolen or created by very low paid workers, which is exactly what has happened with AI (go look at markets for training data in Kenya and the Philippines for example).
My favorite example is towel folding robots. Everyone these days trains a robot to fold towels. How? They pay thousands of people in poor countries to video themselves folding mountains of towels all day lol. Or the cleaning robots where people are offering human cleaning services for free so they can record a human cleaning functionally every home in a city to train a robot to clean those homes. Or Elon musk training self driving on more miles of video than any human on earth could ever even drive in a lifetime.
In the end, they will probably fold more towels training the models than they will ever fold for a customer. A human folds something like 20 towels a month on average, so when they have trained on video of 100T towel folds we know they might be close to selling a statistical towel folder.
If the way I’m describing AI makes it sound stupid, it’s because AI is actually really fucking stupid.
Everyone thinks I’m being a curmudgeon as a bit to make a point. I’m simply explaining reality.