I train a model whose pretraining is partly synthetic. Nobody hands you clean human text for the kind of work I care about, which is training a model to follow a long tool sequence inside a messy codebase. Behavior like that barely exists as recorded text online. It has to be produced, and the most direct way to produce it is to have a model attempt the task, write down what it did, and train the next version on the transcript. So the loop people write warnings about is not an abstraction to me. When I read the collapse literature for the first time I recognised the shape of my own pipeline in it, and it sent me back to my training logs to look for what my model had been quietly losing.
Correctness holds up the longest. Long before accuracy anywhere near moves, the range narrows. A model that once reached for ten different ways to solve a problem drifts until it reaches for one, then a polished version of that one. The failures you usually ship software to catch, wrong answers, bad code, are the last things to appear. What disappears first is the set of things the model is willing to try, and no unit test, no benchmark, no eval I had in place was measuring that.
That gap, between the thing everyone measures and the thing that actually rots, is the whole problem worth thinking about.
The silent mode of collapse
The literature gives this a name and a shape. When recursive training strips the tails of a distribution, it does not do it evenly, and the early work made that explicit. Shumailov and colleagues, in the paper that ran the effect through working language models, described defects that do not repair themselves and tails that disappear. Their earlier work had shown the mechanism across several model families. What I added by living with a real pipeline is a sense of which mode you actually encounter in practice.
The collapse that shows up as an error, a garbled generation, is the loud kind. It gets caught, it gets fixed, it gets a postmortem. The collapse that matters is the other one, where the model keeps producing acceptable, fluent, on-spec output while the distribution it samples from quietly tightens. Alemohammad and colleagues split exactly this into quality and diversity and showed you can lose one while holding the other steady. That asymmetry is what I want people to sit with. A decaying system that still answers correctly within a narrowing range will pass almost every check you run, because every check you run measures quality, and quality is the thing that is not falling yet.
I suspect this is why so much of the field slept on collapse for years. We built monitoring around the loud mode, errors and surprise, because those are visible and embarrassing. The silent mode produces no anomaly. It produces a slow convergence toward the most average thing the model has ever seen, and the average is exactly what looks fine.
The boundary condition I keep moving
Here is the connection that changed how I build.
I wrote once about physics-informed neural networks, and the boring, correct point in that post was that boundary conditions are constraints, not optimization targets. When you fold a hard requirement into the loss as a weighted penalty, the optimizer is free to quietly trade it away against the other terms. The fix was to bake the boundary into the architecture so it could not be violated. That idea transfers almost exactly to synthetic data.
Most teams treat the amount of fresh, real, human data in a training run as a knob on a loss, something to be weighted and tuned alongside everything else. That is how it quietly drifts toward zero. The more useful framing is to treat real data as a boundary condition, a hard floor that the pipeline is structurally not allowed to go under, the way the architecture could not violate its boundary.
I started treating it that way in my own systems. On the training side I kept a set of genuinely human, in-domain samples that is never mixed into the recycled pool, and I check the model against it on a schedule, treating it as an anchor the way you would treat a held-out test set you are forbidden from overfitting. On the generation and inference side I have a different anchor: a layer of deterministic signals that are checked before anything is shipped. Sentinel, the piece of my stack that does this, runs hundreds of fixed, explicit checks and only then lets a model’s output go through. That is a boundary condition for a self-generated loop, a way of forcing the produced text, which would otherwise be purely the model talking to itself, to touch something outside the model before it is trusted.
None of this removes the need for synthetic data. It removes the part where synthetic data becomes the only input.
The ratio is the real scarce resource
The formal result that most people quote on this is that training on synthetic data alone cannot avoid collapse, while mixing in a bounded amount of real data can. Seddik and colleagues made that precise in a statistical model of language training. I read it as a statement about scarcity rather than a warning about a specific technique.
What is scarce is not compute, and not model quality, and not even human data in some absolute sense. What is scarce is the ratio, the share of fresh, external, real-world signal relative to recycled output, that keeps a loop from feeding on itself. You can buy compute. You can generate unlimited synthetic text. You cannot generate the part of the input that resists the loop, because that part is defined precisely by not being produced by the loop. Every lab that started licensing and commissioning rights-cleared human data reached that conclusion on their own, through economics if not through reading the math.
The flywheel that everyone wanted, a model that trains on its own output and climbs forever, turns out to be a pipe dream only if you close it. The flywheels that actually work keep a tap of fresh input open, and treat that tap as the thing you protect, not an expense to minimise.
The same shape in the places I work
Once the mechanism is clear in models, I stop being able to unsee it in organisations, which is where I spend most of my working life.
I run a recruiting platform, and I watch the mirror problem reproduce there at the speed of a reporting chain instead of a training run. The candidate data that reaches a hiring manager has already been filtered through a recruiter, an amount of formatting, a round of reassurance. The angry candidate, the signal that the process is broken, gets softened into “the account needs attention” before it arrives anywhere it could change a decision. Everyone involved is being accurate, defensibly accurate, at each step, and yet the organisation re-learns a narrower version of itself every quarter. The rare signal, which is the same signal as the uncomfortable signal, is the first thing to be averaged away. It is the tail.
I believe the collapse result is the cleanest statement we have of something that is true in a lot of places and was never confined to machines. Any system that learns mostly from its own output, its own filtered history, its own comfortable stories, will gradually forget the parts of the world it did not keep. The uncomfortable part is that none of this requires anyone to be dishonest. Each edit in the chain is defensible on its own. That is exactly why it is so hard to catch, and why the decay reads, from the inside, as the opposite of decay.
What I actually do about it
Working with models, the practical version reduces to a few things I now hold as rules.
Measure range, not just correctness. I add a cheap proxy for linguistic or behavioural diversity to evaluation and log it across training. Collapse is a narrowing, and narrowing is invisible unless you have a metric that tracks how wide the output still is. Most teams have accuracy, loss, and a dashboard, and none of the three records the thing that is actually degrading.
Keep an untouchable anchor. I hold out a set of real, human, in-domain samples that never enters the recycled pool, and I treat it as a boundary condition, on the same footing as the requirement that the physics solve the boundary. It is the reference that tells me whether the model is drifting toward its own average, and without it the drift is silent until it is expensive.
Make the ratio a policy, not an accident. Synthetic data is a normal and necessary part of what I train on, especially for behaviour that does not exist in the wild. The decision that is usually left to drift is how much of the input is allowed to come from the model’s own output. I decide that explicitly, and I revisit it, because the default is for it to creep upward on its own.
Be willing to look slow. The un-collapsed input left in the world is mostly produced by people who answer for what they write, and collecting it is more expensive and far less impressive than scaling out compute. It is also the only input that keeps the loop honest. The unglamorous path is the one that works, which is the reason so few systems choose it.
What I take away
The cheap reading of model collapse is a warning about one piece of machinery, don’t let the models train only on themselves, and it is correct as far as it goes. The version I keep carrying is broader. A system that learns primarily from its own output, and mistakes that output for the world, will gradually forget the parts of the world it did not keep. Not because anyone is careless, but because closing the loop is tidy and efficient and looks like progress right up until the range is gone.
The fix in every setting is the same, and it is a boundary condition, not a tuning knob. Keep a channel of fresh, external, real signal open, in a model, in an organisation, in whatever you are building, and treat it as the thing you are not allowed to trade away. The mirror reflects what is already in the room. The world is the only thing that shows you something else.