Tag: information-theory
All the articles with the tag "information-theory".
The Mirror Problem
Published: at 03:44 PMA model trained on its own output loses range generation after generation, the tails of its distribution slowly disappearing. I hit this kind of loop in my own systems, and it changed how I think about the difference between a model learning and a mirror reflecting.