@MiTiBennett Outsider to this field, but I thought the Invariance Principle justified MDL under the argument that description length should translate across universal compilers. What’s the refutation to this?
@mathelirium Though if one zooms in far enough, no physical instrument can detect what happened during a dt. Real instruments can only observe the macroscopic, cumulative effects, not the infinitesimal fluctuations.
This is what a moment of learning looks like!
Process called synaptogenesis. This is the formation of new connections, known as synapses, between neurons in the brain.
The blueprint for this "grand unification" already exists:
---------
🔹1961: Landauer established the thermodynamic cost of bit erasure (https://t.co/JIY1djdrRX)
🔹1982: Bennett resolved Maxwell’s Demon, proving that logical irreversibility (erasure), rather than measurement, necessitates thermodynamic work (https://t.co/fwtWHCsm5N)
🔹Late 1990s: Jarzynski and @gavincrooks related non-equilibrium work to equilibrium free energy. Their Fluctuation Theorems were later used to derive the entropy production cost of statistical inference (https://t.co/4J6G1aU1Qb; https://t.co/qRvf7ZMlac)
🔹2007: Kawai et al. proved average thermodynamic entropy production equals the KL divergence between forward and backward path distributions( https://t.co/DyzcIURZWQ)
🔹2009: Sagawa & Ueda incorporated information theory into this framework, bounding the energetic cost of measurement and feedback (https://t.co/VtDFAB8IgV)
🔹2019: Wolpert extended stochastic thermo to formal computational architectures, including Turing machines and boolean circuits (https://t.co/kfWTE5Z4nL)
🔹...and many more works in thermo/non-equilibrium stat mech that are waiting to be formally connected to computer science/ML concepts.
---------
Formalizing this mathematical connection is a central research interest of mine. Reach out if you have ideas (DMs open).
Excited about self-organising sytems? Do you have a cool paper, either in the works or ready? Then consider applying to our Evolving self-organisation workshop at @GeccoConf ! Submission Deadline March 27
Also check out the amazing workshop website:
https://t.co/8gXc8CERWS
I increasingly think so much intelligence is embedded in the connectome.
I wrote about how this connects with AI-generating algorithms: what we need are connectome-generating algorithms.
https://t.co/gIdivs7ton