@LepoulpePoulpo@TimothyDuignan Shannon’s coding channel thm. Central to digital communications. Showed that almost perfect signal could be transmitted w/o needing excessive redundancies (3-5G networks, WiFi etc)
Not a result but the notion of “cross entropy” —every GPT is pretrained with that loss function.