@shelbystardust If science/medicine are real, why does location predict literacy/life expectancy? Truth still spreads through culture, which is always localized. Of course that doesn’t automatically confirm religion, but doesn’t disconfirm it either.
Any time we train an ML system, we want to achieve generalization: for the model to perform tasks on data it hasn’t seen before. Going back to image classification, we want the system to recognize, for example, a dog it has never seen before. Even though its output may be just a label (“cat” or “dog”), training encourages it to learn features that make dogs recognizable across different images.
In a similar way, an LLM is “just” trained to predict the next token (word). But *truly* predicting the next word across many different situations encourages it to learn grammar, syntax, semantics (meaning), and patterns of reasoning. These abilities can then be applied to situations it has never seen before.
In a sense, humans do this as well. We have only a finite amount of experience, and our evolutionary history certainly didn’t specifically prepare us to discover quantum mechanics. It prepared us to throw rocks and find food, for which it’s useful to reason and understand physics (in a however confused way at first). Yet, these abilities remarkably generalized.
Here is Ilya Sutskever, explaining it as well: https://t.co/HNtoWSZLBy
@Be_like_legend They scaffold working memory for you. Their mere presence suggests a lot of things: "I am A, here with my friend B, doing C". These are encoded a thousandfold more efficiently by their social and body cues, than in your conscious mind. Also, implicit emotional regulation.
@benmoha@polynoamial@OpenAI You do realize that this describes a genie or a God at some point, right? At what point did a box which does exactly what you ask it to do, grants you a wish, in increasing order of complexity, become normalized??