Real data is finite, expensive to label, and often tangled with privacy obligations. Synthetic data — examples generated by a model rather than collected from the world — promises to break all three constraints at once. It is already used to train perception systems on rare driving scenarios and to share medical data without sharing patients. But a subtle failure mode lurks in the approach: when a model trains on data produced by another model, the tails of the distribution can quietly disappear.

Synthetic data buys scale and privacy by sampling from a learned approximation of reality, and the approximation is always narrower than the thing it approximates — the missing tails are where rare failures live.

Part One: What is being compared

Real data is measured. A photograph is a photograph, a transaction happened, a LiDAR return came from a physical surface. It carries the full messiness of the world: the rare edge cases, the label errors, the sensor glitches, the long tail of circumstances that matter precisely because they are unusual. Its limits are cost, scarcity of rare events, and the legal and ethical constraints on collecting and sharing it.

Synthetic data is generated. A simulator or a generative model produces examples with known labels — a self-driving car in a snowstorm that was never driven, a lesion with a confirmed diagnosis, a customer who never existed. Labels are free and perfectly consistent, rare scenarios can be oversampled deliberately, and no personal information leaks because no person was involved. Its limit is fidelity: it is only as good as the model or simulator that made it.

The comparison is between the messy truth and a clean approximation. One is authoritative but scarce; the other is plentiful but second-hand.

Part Two: The arithmetic

The economics favour synthetic data at the front end. Labelling a single high-quality image for object detection can cost several dollars and take minutes of human attention, while generating a labelled synthetic image costs a fraction of a cent in compute — a one-to-two-orders-of-magnitude gap at dataset scale. For a perception task needing millions of examples across rare conditions — night, glare, occlusion — synthetic generation can cut dataset cost by one to two orders of magnitude. That is why autonomous-driving stacks and robotics teams were the first adopters.

The sim-to-real gap is the tax. A model trained only on simulator images often performs well in simulation and badly on real hardware, because the renderer's textures, lighting and sensor noise differ from physics. Teams bridge it with domain randomisation — varying every simulateable parameter so the real world looks like just another variation — which helps but does not guarantee coverage of phenomena the simulator never modelled.

The deeper problem appears in recursive training. Research on model collapse shows that if you train a new generation of models on data produced by the previous generation, without mixing in fresh real data, the distribution narrows with each round. Low-probability events vanish first, because they are the least likely to be sampled into the training set. After several generations, the model's output becomes a smoothed version of the original distribution — plausible on average, but missing the tails. In one line of experiments, a model trained purely on its predecessor's samples lost the rare modes entirely within a handful of generations, even though average accuracy looked acceptable.

The practical number matters: collapse is slow when each generation keeps a substantial fraction of real data and fast when real data is dropped. This is why the useful pattern is not "synthetic instead of real" but "synthetic mixed with real," where synthetic data fills in rare cases and real data anchors the distribution.

Part Three: Where the model breaks

The arithmetic assumes the generator is good enough to be a faithful stand-in. For some tasks it is. For others — anything involving human behaviour, subtle social context, or poorly understood physics — the generative model's errors become systematic biases in the training set, and the downstream model learns the bias as if it were truth. A synthetic customer who never existed can still encode the generator's assumptions about what customers are like.

Privacy claims also need care. Synthetic data is often sold as inherently private because it is not about real individuals. But a generative model trained on real records can memorise them, and sufficiently detailed synthetic records can sometimes be matched back to real people — a risk known as memorisation and membership inference. Strong privacy requires formal guarantees, such as differential privacy during generation, not just the absence of names.

There is a measurement trap too. Synthetic data is usually evaluated by how well a model trained on it performs on a real holdout set. If the holdout is small or unrepresentative, the evaluation misses exactly the tail failures that caused the problem. Teams report excellent aggregate accuracy and discover the gap only when the system meets an unusual case in production.

Part Four: The later cost

The maintenance cost is easy to underestimate. A synthetic pipeline is a piece of software that must be kept in sync with the model it serves. When the generator is updated, the distribution of training data shifts, and every downstream model may need retraining and revalidation. Real datasets also change, but slowly and for visible reasons; synthetic datasets can change on a Tuesday because someone tuned a sampler.

Verification becomes a discipline of its own. Because synthetic labels are generated rather than measured, the usual assumption that the label is ground truth is false. Teams must audit generated labels, check coverage of rare cases, and maintain a real-data benchmark that is never replaced by synthetic approximations. That benchmark is the anchor that keeps the system honest.

There is also a regulatory dimension. In domains such as medical devices or autonomous vehicles, regulators may require evidence from real-world data regardless of how good the synthetic results are. A company that replaces all real testing with synthetic cannot necessarily certify its product, which limits the cost savings in the most regulated — and highest-value — applications.

Part Five: Outside the team

For the person using the product, the difference shows up in rare situations. A voice assistant trained with synthetic speech may work beautifully for the accents it generated and fail on a speaker with an unusual voice — a failure that feels arbitrary because it is. A medical screening tool trained on synthetic lesions may miss the atypical presentation that did not make it into the generator's output.

For the engineer, the choice decides what the test set must contain. With real data, the test set is a sample of collected examples. With synthetic data, the team must deliberately construct a real holdout that the synthetic pipeline never saw, because otherwise the evaluation is circular — measuring a model against data made by the same process that trained it.

For an organisation, the strategic question is where the real-data anchor lives. A team that keeps a small, high-quality, continuously refreshed real dataset and surrounds it with synthetic augmentation gets the best of both: scale from generation, truth from measurement. A team that drops the anchor saves money up front and pays for it when the distribution shifts in a way no generator predicted.

A fair comparison fixes the task, the real holdout, the rare-event requirement and the privacy constraint, then measures performance on the rare tail — not just average accuracy. Comparing synthetic convenience against real-data accuracy on the same easy majority class is how the collapse goes unnoticed.

Market sizing check

If generative models keep improving and data licensing costs keep rising, the market for synthetic data generation and the tooling around it — simulation platforms, privacy guarantees, validation suites — is plausibly $6B to $15B a year by 2030. The swing factor is whether model collapse can be reliably avoided through mixing and anchoring, and whether regulators accept synthetic evidence in high-stakes domains.

These figures are a synthesis of published market ranges and a stated model, not a forecast of any specific vendor.

The comparison that matters

The comparison that matters is between scale and fidelity. Real data wins when the tail matters, the domain is poorly modelled, or regulation demands measured evidence. Synthetic data wins when rare cases must be manufactured, labels are expensive, and privacy rules make real sharing impractical.

The ranking assumes the generator stays close to reality. Introduce recursive training without a real-data anchor and the synthetic advantage inverts: you get unlimited data from an ever-narrower approximation, and the failures cluster exactly where the data no longer looks real.

Synthetic data is a multiplier on real data, not a replacement — it scales the cases you understand and quietly deletes the ones you do not, which is why the rare tail must always be anchored in something measured.

SOL: HCqPmPJM3NVtS6YrmVaTZYnuP8XZ2PFgQZKT94VK1LkS