What Is Synthetic Data? Can It Replace Real Web Data?

Published:

15 minute read

Acar Diveroli
Written by: Acar Diveroli
A small blue REAL SAMPLE cluster feeds a dense white GENERATOR core, which spins out a wide cloud of thin lines and points

A developer needs ten thousand customer records to test a new billing system, but the real ones are not allowed to leave the production database. A machine-learning team has only a few dozen examples of the fraud pattern it wants to catch. An AI team wonders whether it still has to collect web pages when a language model can write training examples on demand. All three are looking at the same option: synthetic data, data that a program produces instead of data that someone collected.

This post covers what synthetic data is, how it is generated and checked, its main types and uses, what it cannot do (model collapse included), how it compares with real web data and what it means for privacy.

What is synthetic data?

Synthetic data is data generated by a program to resemble real data, rather than captured from real transactions, sensors, people or web pages. NIST, the US standards institute, describes the process in its de-identification guide SP 800-188: seed data are used to create artificial data that keep some of the seed data's statistical characteristics.

There is no single agreed definition. A survey commissioned by the Royal Society proposes a broad one: data generated with a purpose-built mathematical model or algorithm to solve a data science task. That covers invented customer tables, reviews written by a language model and street scenes rendered by a 3D simulator. The idea goes back to Monte Carlo simulation in the 1940s; generative AI has made it far more common.

NIST's guide also separates degrees of "synthetic":

  • Partially synthetic data keeps the real records but replaces the risky cells, such as an unusual income, with modeled values or noise.
  • Fully synthetic data is generated in full from a model, so no row maps one-to-one to a real record.
  • Test data copies only the structure and value ranges of a dataset, for software testing, with no attempt to match its statistics.

How is synthetic data generated?

Synthetic data is generated by fitting a model, writing rules or building a simulation, sampling new records from it and then checking those records against real data. A typical project runs in seven steps:

  1. Define the job. Decide what the data must support: testing a form, training a classifier, sharing a table with a partner. This decides which properties have to survive and which can be dropped.
  2. Pick the starting point. That is a real sample (the seed data), a set of domain rules such as valid postcodes and price ranges, or a simulation of the process itself, which needs no real records at all.
  3. Choose the generator. Options range from simple rules and random values, through statistical models that learn distributions and correlations, to machine-learning generators: generative adversarial networks (GANs), variational autoencoders (VAEs), diffusion models for images and large language models (LLMs) for text.
  4. Generate. Sample as many rows, texts or images as needed, and mark them as synthetic inside the data itself. NIST suggests obviously artificial values such as "FIRSTNAME1 LASTNAME1", so nobody mistakes a test file for real customer data.
  5. Validate against real data. Compare distributions and correlations with the source (fidelity), and check whether a model trained on the synthetic set still performs on real, held-out data (utility).
  6. Check privacy. Look for generated records that sit too close to real ones, and test whether outliers can be traced back to a person.
  7. Document and keep a real test set. Record how the data was made, and keep real data aside for the final evaluation.

Validation is the step that decides whether the data can be trusted. The US Census Bureau publishes the SIPP Synthetic Beta, a synthetic version of a household survey linked with tax and benefit records. Researchers develop their analysis on the synthetic file, then Census staff run the same programs on the confidential data and review the output before it is released.

What types of synthetic data are there?

The main types differ by the shape of the data and by how it is generated:

TypeHow it is madeTypical useWatch out for
Tabular recordsStatistical model, GAN or VAE fitted to a tableData sharing, analytics, fraud modelsBroken correlations, copied outliers
TextA language model writes examples from promptsTraining and testing chatbots, classifiersFactual errors, repetitive phrasing
Images and video3D simulation or image generatorsSelf-driving, robotics, defect detectionGap between simulated and real scenes
Software test dataRules and fake-value generatorsQA, demos, load testsMistaken for real data

Text is where the question "can it replace real web data?" comes up most. In published work, generated text is usually added to filtered real text rather than used instead of it. Microsoft Research's 2023 paper "Textbooks Are All You Need" trained a small code model, phi-1, on about 6 billion tokens (word pieces) of "textbook quality" web data plus about 1 billion tokens of textbooks and exercises generated with GPT-3.5.

Why do companies use synthetic data?

Companies use synthetic data when real data is scarce, sensitive, expensive to label or does not exist yet:

  • Privacy. A bank or a hospital can give developers and vendors a synthetic copy of a table instead of customer or patient records, with the caveats covered below.
  • Rare cases. Fraud, machine failures and dangerous road situations are rare in real data; generating more examples helps a model learn them.
  • Software testing. A new app needs realistic-looking accounts and orders before it has users. Test data only has to be valid, with the right formats and ranges, not statistically faithful.
  • Labeling cost. A simulator already knows where every car and pedestrian in its scene is, so the labels come free; labeling real photos by hand is slow.
  • Balancing. Generated examples can fill out under-represented classes, which helps only if the generator itself is not biased.

What are the limits of synthetic data?

Synthetic data cannot add information its source never had: a generator reproduces patterns from its training data, rules or simulation, and nothing beyond them. NIST's guide says it directly: fully synthetic data cannot give the user more information than the model it came from.

This shows up in several ways:

  • No new facts. A generator trained on last year's prices produces plausible last-year prices; it knows nothing about this week's price cut or yesterday's product launch.
  • Lost outliers. A generator either smooths rare, extreme values away, so the data looks more average than reality, or copies them closely enough to expose the real record.
  • Copied bias. If a group is under-represented or treated unfairly in the source, the synthetic data inherits that and can make it look like neutral data.
  • Fluent errors. LLM-generated examples can be confident and wrong. Text that reads well still needs fact checking.
  • False discoveries. A pattern found in synthetic data may be an artifact of the generator, which is why NIST advises analysts to confirm findings on the original data.

What is model collapse?

Model collapse is the loss of quality that appears when generative models are trained, generation after generation, on content produced by earlier models. Shumailov and colleagues described it in a 2024 Nature paper, reporting that "indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear". They showed the effect in language models and in two other kinds of generative model.

In their language-model experiment, each generation of a small model was fine-tuned on text written by the previous generation. With no original data kept, performance on the original task fell; in a setting that kept 10% of the original human-written data, the degradation was minor. The lesson is not that every synthetic example is harmful, but that the rare cases, the tails, disappear first when models feed only on their own output.

The authors also warn that training new models may become harder without access to data crawled from the internet before AI-generated content spread widely. Open crawl archives such as Common Crawl keep dated copies of public pages, which is one reason they matter.

How does synthetic data compare with real web data?

Real web data records what a public page actually showed at a given time; synthetic data records what a model expects. The difference matters most for freshness and for ground truth, the verified real values a model is measured against:

QuestionSynthetic dataReal public web data
How fresh is it?As old as its sourceAs new as the last collection
Is it ground truth?No, it reflects a modelYes, for what the page showed
New prices, products, eventsNot includedIncluded once published
Rare cases on demandYesOnly if they occurred
Personal data riskLower if done well, not zeroPresent; minimize at collection
Location-specific detailOnly if the source had itDepends on where you collect from

Why do AI teams still collect real web data?

AI teams still collect real public web data because evaluation, grounding and freshness all need facts that a generator cannot invent.

Evaluation. The Royal Society-commissioned survey recommends that tools meant for the real world be evaluated, and if needed fine-tuned, on real data. A synthetic test set mostly measures how well a model imitates the generator.

Grounding. Many AI products answer with retrieval-augmented generation (RAG): they fetch current documents at question time and let the model answer from them. That only works if the documents are real and recent, which is the pipeline described in Web Scraping for RAG.

Freshness and seeding. New products, prices and regulations appear on the web first, and even synthetic text pipelines usually start from real examples. The model-collapse research shows why some real data has to stay in the mix.

Is synthetic data anonymous?

Synthetic data is not automatically anonymous: if it was generated from personal data, it can still reveal information about the people in that data. The Royal Society-commissioned survey calls the belief that synthetic data is inherently private "a common misconception" and notes that generators can memorize and reproduce their training records.

NIST makes the same point: both the models and the synthetic data they produce may leak personal information that can be re-identified. A common remedy is differential privacy, a mathematical method that limits how much any one person's record can change the output, at the cost of some accuracy.

The legal view follows the same logic. Under the GDPR in the EU and KVKK in Türkiye, data counts as anonymous only when it can no longer be linked to an identifiable person, and building a generator from customer records is itself processing of personal data. If your source data came from scraped pages, How to Handle Personal Data in Scraped Datasets explains the difference between masking, pseudonymization and anonymization.

Where do proxies fit when you need real data?

When a model has to be grounded or tested on real, current public data that differs by location, such as prices, listings or job ads, synthetic data cannot supply it, and the data has to be collected from the right place. Many sites show different prices, currencies, stock and languages depending on the country of the visitor's IP address, so collection from one office sees only one version.

A Residential Proxy sends each request through a home connection in the country and city you choose, so the page returns what local visitors see. Collect public pages only, follow robots.txt and the site's terms, keep request rates low, use an official API where one exists, and leave out personal data you do not need.

Teams that would rather not run the collection themselves can talk to our sales team about managed page extraction through our Web Scraping API.

Common mistakes

  • Calling synthetic data anonymous by default. Test it for leakage, or use a method with a formal guarantee.
  • Validating by eye. "It looks real" says nothing about correlations, rare cases or utility.
  • Testing the final model on synthetic data only. Keep a real, held-out test set.
  • Leaving records unlabeled. Synthetic rows that look real end up in reports as if they were real.
  • Training on model output in a loop. Each generation without fresh real data loses more of the rare cases.
  • Expecting a generator to know recent events. Anything after the source was collected is missing.
  • Ignoring bias in the source. The generator copies it, then presents it as clean data.

Decision guide

NeedUse
Test an app before launchRule-based test data, clearly labeled
Share a customer table with a vendorSynthetic data with privacy testing, plus legal review
More examples of rare casesSynthetic augmentation, evaluated on real cases
Teach a chatbot a task formatLLM-generated examples, checked by people
Current prices, listings or job adsReal public web data, collected politely
Answers about recent eventsRetrieval from current real sources
Final model evaluationA real, held-out test set

Frequently asked questions

What is an example of synthetic data?

A table of 10,000 invented customers whose ages, cities and purchase amounts follow the same patterns as a real customer database is synthetic data. So are product reviews written by a language model to train a classifier, and road scenes rendered by a driving simulator with every car already labeled.

Is synthetic data reliable?

It is reliable for the job it was validated for, and not beyond it. Data that passes fidelity and utility checks for one task, such as training a fraud model, may still mislead in another, such as estimating rare events. Treat findings from synthetic data as hypotheses and confirm them on real data.

What is the difference between synthetic data and mock data?

Mock or dummy data only has to look valid: the right formats, plausible ranges, no real people. Synthetic data in the stricter sense also keeps the statistical patterns of a real dataset, so analysis or model training on it gives similar results. Mock data suits software tests; statistically faithful synthetic data suits analytics and machine learning.

Can AI models be trained only on synthetic data?

Some narrow models can, especially in simulation-heavy fields such as robotics, but large language models still depend on real text. Published work mixes the two, as phi-1 did, and the model-collapse research shows that training on generated text alone, generation after generation, degrades quality.

What is synthetic data in market research?

In market research, synthetic data usually means "synthetic respondents": a language model answers survey questions the way it predicts a type of customer would. It can help draft and test a questionnaire, but it cannot report what real people think today or how they react to something new, so decisions still rest on real surveys and on real market data such as competitors' prices and listings.

How do you measure the quality of synthetic data?

Quality is measured on three axes: fidelity (how closely distributions and correlations match the real data), utility (whether a model trained on it performs well on real data) and privacy (whether real records can be inferred from it). A strong result on one axis often costs something on another, so set the targets from the use case.

Summary

Synthetic data is generated by a model, rules or a simulation to resemble real data without recording real events or people. It helps with software tests, safer data sharing, rare cases and training examples, provided it is validated against real data and checked for privacy leaks. It cannot replace real data: it holds nothing its source lacked, misses everything that happened later, copies bias and degrades models that feed on their own output. For current, location-specific facts such as prices, listings and job ads, the source is still the real public web, collected at a polite rate and from the right country.

Ask ChatGPTAsk Claude