I made this fun side project and discovered many interesting self organizing patterns! The code is less than a page (excluding GUI). #artificial#life#simulation
@fchollet I think the word is modularity. Modular software has more boilerplate code at the expense of scalability. Transformers are more modular than naive NN, hence, more scalable. Modularity = Generalizability.
@victortrac@Andercot Even with a universe-sized computer, you can only predict the next state of the universe or whatever is the smallest time-step update (you can't jump to any point in future because of computational irreduciblity)
@Plinz Thanks for your input. This also reminds me of another topic which I think is related - The Localist vs. the Distributed representation of connectionist models. The latter looks to be the winner in today's approach for AI.
@elonmusk But why hate did evolve in the first place? It must have some survival value in some situations especially where fear is involved. So beating hate = beating fear
@ilyasut Intelligence evolved later in life. So yes there are many other animal qualities have a much higher value for the survival in evolutionary terms.
@tegmark But LLM's are trained on Wikipedia and Wikipedia pages have the coordinates of every city. It would be interesting if the model can predict the coordinates of locations not in their training data.
@Plinz The feeling of continuity might be an illusion in an infinite possibility universe. Even If the universe change its state randomly every second, we would still feel continuity if we have false past memories in each of the independent random state changes.