The whole purpose of research in conversion optimization is to either increase the sample size of people contributing to a website or to improve the quality of that sample.
Brian explains: As a marketer, I’m familiar with the customer. I’ve read the marketing research. I’ve studied the personas at some point, but I’m going to be writing this landing page with a sample size of one. I’m going to be the only one contributing to it.
So I’m going to put onto the page the things that I think are most persuasive or the things that the visitor is looking for. I’m going to be the final arbiter of that. Now, if I ask a designer to come in and take a look at that or tell me about that, we’ve increased the sample size to two. Two of us are providing input on this creative exercise. If I get the executive team and other people on it, I might be able to expand the sample size to 10.
Statistically, that’s an incredibly small sample size. And we are what the Eisenberg brothers called “inside the bottle”. When you’re inside the bottle, inside the business, you can’t read the label. So you’re blinded by what you believe to be true rather than being able to look at your business or product as your visitors are looking at them.
With conversion optimization, we want to increase the sample size. We want 25 or 50 or 100 people outside the business to take a look at the creative design. And that’s where these user research tools come in.
Brian’s team analyzes the typical design process, which because of small sample sizes and poor input from external sources from prospects and customers, is not serving you well. Then they will increase the sample size and the quality of participants, as well as undertake research. That gives Brian indications that he’s going to be able to improve the website with this new creative effort.