Skip to content To unlock the full white paper click on the link below.
Last year, we took one of the most talked-about ideas in market research and put it to a real-world test. Could synthetic data really stand in for human responses? We took thousands of real survey responses, asked synthetic data providers to recreate the people we held back, and measured – metric by metric – how close the artificial answers came to the real ones.
The picture was mixed. Synthetic data was good at producing individual answers that looked plausible, but poor at producing people who behaved consistently across a whole survey. It could mimic a topline result but it could not yet be trusted with the deeper analysis on which real decisions rest.
A year is a long time in AI. The models have moved on, one of the providers has had direct feedback from our first test, and the noise around synthetic research has only grown louder. So we ran the study again, using the same survey framework and the same standards of rigour, to answer a simple question: Has synthetic data actually got better, and is it ready for the work that matters?
Synthetic data is artificially generated information designed to mimic human responses. Put simply, we were testing whether AI-generated responses could reliably stand in for the answers you would normally collect from real people. The models are trained on genuine responses and try to recreate the patterns and relationships found in that original data. Unlike weighting, which simply makes existing responses count for more, synthetic data creates brand-new data points that were never collected.
It is worth drawing a line early, because the term is used loosely. This study is about synthetic augmentation – using artificial responses to ‘boost’ a real sample that is too small. That is not the same as ‘synthetic persona’ and ‘digital twin’ tools, which are designed to simulate consumer conversations rather than complete structured surveys. The two are often talked about as if they were one and the same. They are not, and that distinction matters if you want to use them responsibly.
Synthetic augmentation (this study). Artificial respondents added to a real quantitative sample to fill gaps. Usually, a substitute for interviewing more people.
Synthetic personas / digital twins. Conversational tools that simulate how a group (often called twins) might think or react – useful for rapidly pressure-testing ideas. Usually a complement to interviewing people, not a stand-in for the data.
The pressure that makes synthetic data attractive is real. Insight teams are being asked to prove ROI, to move faster, and to spend less – all at once. Certain audiences remain genuinely hard and expensive to reach: niche B2B professionals, specific patient groups, smaller demographic segments, and younger cohorts. At the same time, synthetic data providers, many of them venture- and private-equity-backed, are taking the message directly to senior decision-makers – promising faster, cheaper insight, with the expectation that insight teams will make it work.
That is exactly the environment in which a calm, evidence-led answer is most useful. The honest test is not whether synthetic data can produce a number, but whether that number can be trusted enough to act on – and what it would cost to be wrong.
And there is a blunter way to frame it. After years of hype and heavy investment, the fair question is no longer whether synthetic data is promising, but whether this is as good as it gets.
As last year, we focused on practical questions that matter to insight teams and decision-makers:
We judged the providers against five success criteria defined by STRAT7: accuracy, consistency, variation, key drivers and segment grouping. These are not the same measures providers tend to use when assessing themselves. They are tougher tests, designed to reflect whether synthetic data can be trusted in real insight work.
Last year, synthetic data showed promise but clear limitations. This year, we wanted to know whether the technology has improved enough to change that conclusion.
Three new dimensions. We added a Van Westendorp pricing exercise and willingness-to-pay questions; a topical situational question (whether the 2026 World Cup makes people more likely to buy snacks); and – most importantly – a tracking test, comparing real change between 2025 and 2026 against the change the synthetic data implied.
A first, to our knowledge. This makes ours the only study to test synthetic data vendors at two points in time using the same survey for comparability – a genuine like-for-like read on whether the technology is improving.
Three terms recur throughout the study. Each describes a different role in the test.
The portion of genuine responses we handed to the providers to build their models. In the study it also stands in for the incomplete, real sample a researcher is trying to top up.
The synthetic responses the providers generated to supplement – or ‘boost’ – the real data. In a live project, this would be blended with training set responses to reach the sample size or representation a study needs.
Real responses we deliberately withheld and never showed the providers. Because it is real but unseen, it is the benchmark we measure the boost against. The closer the boost matches the holdout, the more reliable the synthetic data.
In a real project you would never hold data back – you would use every response you had to train the model. That creates an unavoidable problem: in the field, there is no ‘known’ answer to check the synthetic data against.
You are relying on confidence built in advance, from studies exactly like this one, rather than validating results project by project. That is precisely why testing it carefully – and repeatedly – matters.
We surveyed around 3,000 real, nationally representative UK respondents about their snacking, confectionery and crisp buying – awareness, purchasing, attitudes, price sensitivity and brand perceptions. The sample was drawn from an established Shopper Thoughts Research community and put through market-leading data-integrity checks. That combination matters: it means we are almost certainly comparing humans with AI, rather than AI with AI – a genuine gold-standard benchmark.
Much of the analysis focuses on women aged 18-54, used as a realistic proxy for the kind of audience a project might need to boost. In the real world you might want to top up a narrower group still, such as younger shoppers. But to run the test properly we needed enough genuine respondents to hold a meaningful set back. That gave us roughly 640 real respondents in the focus group of interest, split into train and holdout, against which each provider’s synthetic boost was compared. In both cases, the dataset we worked with was around half synthetic – a deliberately demanding test.
We tested two providers using very different methods. To protect commercial confidentiality, we refer to them only as Synthetic Provider 1 and Synthetic Provider 2. Provider 1 was the returning provider from last year and used a statistical, non-LLM method. Provider 2 was new for 2026 and used an LLM-based method trained on market research.
| Synthetic Provider 1 | Synthetic Provider 2 | |
|---|---|---|
| Status | Returning from 2025 | New for 2026 |
| Approach | Statistical machine-learning method (no large language model) | LLM-based, using a proprietary model trained on market-research data |
| In short | Transparent, well-documented, easy to work with | Powerful, but more of a "black box" |
The two providers used different approaches. Each approach is, to a degree, a ‘black box’: we understand the general approach, but not the proprietary detail. For this study, we evaluated them on quantitative questions only, rather than open-ended text. Some question types, pricing and longitudinal comparisons were provider-specific, as noted below.
Has synthetic data finally earned its place in serious research?
Enter your details to get your free copy of the full report, including what we found, key insights, and what this means for you.
