Agreed they're not pure language models anymore. They're models of language plus the objectives and scaffolding we bolted on. So the interesting question isn't "what does language contain" anymore, it's "what do our evals reward." That's a much narrower thing to be a model of, and one we chose.
Currently reading John Gribbin's The Scientists. Such a great and delightful book.
When you read about scientific discoveries they are usually one sentence: "Charles Darwin proposed the theory of evolution." It reads like an instant in time, not decades of painstaking work, and decades more before it by Buffon, Lamarck, Hutton, Lyell and many others whose work led to that moment.
In our age of fast productivity we forget that the most productive moments in history were slow. Deep thought, long conversations, collaboration, and an eagerness to understand the world.
@TheChiefNerd Forgetting arithmetic is fine. Forgetting what the symbols mean is not. Calculators didnโt kill math. Losing the ability to tell a wrong answer from a right one will.
@MelMitchell1 I think the problem is we still do not have a clear understanding of intelligence, consciousness and how they are related. Let alone what the general from of it would mean. Let alone the artificial form of this "General Intelligence".
This book is very thought provoking and great at breaking the perception of isolated western enlightenment. It is super relevant to today's events in the world with routes for energy and goods being the precursor to power and power dynamics.
I highly recommend reading it as it reads really well I could not put it down.
They whole system is the problem. The professors are awarded for quantity and not quality. They will use any means to do that and the publishing system will do anything to support and encourage that for publishing gain. Hence, AI slop papers and AI slop reviews :) if we are pointing fingers we should point it at publishers and reviewers as well.