The day · Page 47
Synthetic users vs real users
For you only
Synthetic user testing is new enough to need one honest page. The tools spin up AI personas and let them loose on your prototype: they click through it, narrate what confuses them, and hand you transcripts and findings in minutes. The example I point people to is uxia.app, a young Barcelona platform built for exactly this: describe the audience (your brief's user line does the job), paste the prototype link, and AI testers walk the flow thinking out loud. Others do a similar job, Synthetic Users was among the first, and the whole space is a year or two old, so expect the details here to age fast.
| Synthetic users (AI) | Real users (humans) | |
|---|---|---|
| Speed | Minutes. Run it between prompts | Days of recruiting for 30 minutes of contact |
| Cost | Pocket money | Incentives, logistics, favours, time |
| On this day | Mid-build, at 11:30, and any time you like | 15:30, once. The main event |
| Best at | Your blind spots: confusing copy, dead ends, the flow only its maker could love | Behaviour. Surprises. The thing nobody in the room predicted |
| Worst at | Being human. It flatters, it invents needs, it feels no friction | Being cheap, fast, or repeatable |
| Trust it for | Arguments for the noon decision | The decision at 16:50 |
| The risk | False confidence, if you treat output as findings | Almost none. This is the honest test |
Why it earns the 11:30 slot: the builder is the worst user of their own prototype, and synthetic testers are surprisingly good at exactly those maker-blind spots. That means the expensive half hour with humans at 15:30 gets spent on real questions, not on discovering that nobody can find the signup button. When the UX agency The Good ran a synthetic platform head-to-head against a real-user study on the same prototype this year, the AI testers surfaced the same top issue the humans found, and also mistook placeholder images for real content. Their verdict fits on a sticky note: "They find the obvious. They miss the subtle."
So hold both truths. Nielsen Norman Group tested synthetic users against their own real studies and landed on supplement, not substitute: AI users flatter, prioritise nothing, and treat every idea as a good one. Jakob Nielsen adds the deeper reason: the point of research is to be surprised, and a model trained to be agreeable will not surprise you. That is exactly how this method uses them. Sparring partner at noon, never the referee. The referee walks in at 15:30.
And you do not need the parallel variant for this. In a classic build, one synthetic run per team around 13:00 catches what the team stopped seeing hours ago.
Synthetic users argue. Real users decide.