Your Survey results are lying. Uncover truth using these statistical methods

Your Survey results are lying. Uncover truth using these statistical methods

Customers are a cryptic bunch. Some use a ton of language to get to a simple point, while some save up on their words and use an emoji or a one word answer which can be interpreted in multiple ways. So your survey results can get really tricky if you start taking responses on the face value because you may not be seeing the complete picture.

The Candy Factory

For instance, you want to find out the effect of a change in process on the consumer behavior. Your product is a fruit flavored candy that sells across the country and you have about and you sell 3 million candies a year. Now your team runs a survey on a population of 500 people in a neighborhood next to your plant.

The Candies from your factory One of your questions asks how frequently someone buys your candy – once a week, once a month etc. Another question down the survey asks about the perception of the customer on quality levels. One more question asks about their age. Let’s review these a bit closely.

A few questions that pop when you look at the responses:

  • Are people who consume my product finding the quality levels to be on the lower side?
  • Are adults more happy overall with my product?
  • Is it true that Kids consume my product more frequently, despite not being too happy with it?

On the face of it, these questions may not be answered by the results you collected. In fact, our brain can trick us into perceiving some of these responses in ways that introduce bias. We have enough bias going into these anyway, so it’s a good idea to let the bias take a back seat while you are analyzing the results. Many of us feel uncomfortable with the idea of using statistics to analyze data and get insights from it. In reality, not all statistical methods are complex, and you have a great deal to learn from your data even if you employ the simplest of them. Here’s some simple techniques that don’t need you to be a Wizard of stats to see and read what your data is trying to tell you.

Is this the right sample?

Your product gets used across the country but your team decided to run the survey on the neighborhood where your plant is located. This does bring in a bias from a couple of angles – one – it only tells you what people in this neighborhood feel about your product. Two – the fact that they live so close to your plant may make them relate to your product in ways that others living far off don’t. Second – the size. For a product that sells in millions, a sample of 500 is too small. Granted, surveys are expensive to run, but you could always run them efficiently using online options, and collect results directly.

Why are there multiple voices in my data?

A simple technique to visualize data to make better sense of it is cross tabulation.

Table 1 – Buying frequency against age

Age Once a dayOnce a weekOnce a monthNot sureTotal
<183253148107
18-30124231590
31-50162119056
>5051214940
Total651287822293

Table 1 here tells us that the younger population dominates the surveyed population. Most respondents (197 out of a total of 293) are under 30. It also shows us that the buying frequency is in general higher – 193 out of 293 respondents say they buy at least once a week. Additionally, we also see that the younger population is buying more frequently – see the cells highlighted in bold (total to 139). The yellow cells form younger respondents who also buy frequently. That wasn’t so complex once we lay the data out on a table, right? Let’s move on.

Table 2 – Quality perception against age

AgeHappyUnhappyNot sureTotal
<1823786107
18-303554190
31-503520156
>50317240
Total12415910293

A similar analysis of Table 2 here tells us the following. A majority of the young population is unhappy over the product quality (132 out of 197) – see cells in orange. While a majority of the older population highlighted in blue (66 out of 96) is relatively happy with the quality. However, if you see the sheer totals, more people are unhappy over the product quality (159) than those who are happy (124).

Table 3 – Quality perception against buying frequency

Buying frequencyHappyUnhappyNot sureTotal
Once a day1841665
Once a week41852128
Once a month5126178
Not sure147122
Total12415910293

Lastly, let’s analyze Table 3 for more insights. Things get a lot more interesting here. We tabulate quality reviews against the buying frequency here. The data tells us that those who buy more frequently (highlighted in orange) are unhappy. Whereas those who buy rarely (highlighted in blue) are rather happy with the quality.

Interesting observations, right? You could use these insights and dig into the root causes of why a certain age population buys more frequently, what are they unhappy about and fix the root causes eventually and run the survey again.

However, you may want to run this on a larger population first and include more locations in it. In fact, running the same survey on a similar population multiple times over a period of time gives you more inputs.

Every time there is a change that is implemented, you could run the survey and see what impact it has on key metrics. Collecting data over a long time also shows you trends in consumption patterns, which could help you further improve your product, communication and distribution.

Hope that was fun.

For more detailed statistical inputs, do check this post on beginner level statistics for researchers. We promise it will be worth it.

Leave a Reply

Your email address will not be published.