People really like quantitative research. You sound really smart when you can say “33% of iPod owners use their iPods less than once a month”, or “17% of white males between 25 and 45 who earn more than 60k a year own a game console”. Numbers are hard to argue with, and specificity implies knowledge.

Quantitative research can be a lot of hard work. There are the basic things you need to do for any quantitative research like:

• write questions
• test them for interpretation
• find enough of the right people to take the survey
• prepare the answers for analysis
• analyze the data
• interpret the results

Each of the steps described above requires expertise to execute well, more expertise than most product managers or entrepreneurs have. Things like how you word a question, whether the response is multiple choice or a scale, how you pick research subjects, how you normalize the data, or how you describe what the numbers say can have an enormous effect how you make a decision. This is what you pay the professional researchers for.

However, even with the hired guns, there are a couple of places where your expert knowledge of the market will determine the ultimate success of the research. Your role in the research is to

• Know what decisions the business needs to make
• Reconcile your market experience with the cold numbers

Knowing what decisions the business needs to make
In the end, once you have your numbers, you still have to make a decision on what to do next. So make sure the survey provides the right ammunition to help you choose one outcome over another. Quant research is too expensive to do just for fun, at least for most people. For each question you should ask:

• What will the outcome of the analysis of this question tell me?
• How will these questions combine to help me make a decision?

If the quant house you are working with can’t or won’t answer these questions, fire them. If you believe the research can help you, and you know why, then it is time to execute the research.

Reconciling your market experience with the cold numbers
One of the great things about quant research is that it translates human experiences, attitudes, perceptions, and other squishy stuff in to hard numbers. Hard numbers are great because we can use mathematical tools like statistics to analyze them. So in a way, quant research allows us to use precise and structured tools in a really messy world. It is critical to remember that the nice clean numbers quant research spits out still live in the really messy world.

What I’m trying to say is that once you get the results back, spend some time thinking about how they relate to the people in the market you have interacted with or customers you already have and how they relate to the research. The research house will provide some color and interpretation of the data, but because you are immersed in the world, you may catch insights they cannot see. This can teach you things like:

• The people you think about when designing the product or business are a small fraction of the market
• There are opportunities outside of the research that still deserve exploration
• There are perceptual or attitudinal aspects of your target customer that are keys to unlocking a market

Quantitative research is a powerful tool when used appropriately. It takes time and money to do well, so make sure it’s going to be useful. Know your part and choose your experts wisely. Once you get the results, do your best to reconcile them with the world you live in. If you don’t know enough to determine if the investment in quant is worth it, consider using qualitative research to build your understanding of the customers so you can make the judgment.