Tag: scaling-laws
All the articles with the tag "scaling-laws".
The Mirror Problem
Published: at 03:44 PMA model trained on its own output loses range generation after generation, the tails of its distribution slowly disappearing. I hit this kind of loop in my own systems, and it changed how I think about the difference between a model learning and a mirror reflecting.
Inverse Scaling Laws in Neural Networks
Published: at 09:19 AMA comprehensive examination of inverse scaling in large language models, where increased computational power paradoxically leads to degraded performance on specific tasks, challenging our fundamental assumptions about AI progress.