Nonergodicity is an important property of real training data, and is arguably responsible for much of the capabilities of LLMs. It suggests a very particular structure for the arrangement of activations in models!
This work was led by @kylejray, with @RiechersPaul and me.
Read more, at our new blog: https://t.co/lR3K0PhMsW
Check out the new blog at https://t.co/6CaeWI2bJM
And @kylejray's beautiful post about the belief geometry associated with nonergodic data https://t.co/hBi2r1e32j!