How we change what others think, feel, believe and do
Doing Mail Surveys
Surveys done using physical mail, or perhaps electronic email, are useful for gathering a large number of views in a way that allows for meaningful statistical analysis.
They are useful for acquiring initial data on which more detailed research is based, including general use of product categories across the market and perceptions of own-brand and competitor products. They are also useful for a later testing of conclusions, for example after concept design and qualitative evaluation with focus groups.
How many respondents do you need? For a typical marketing survey, response rates can vary hugely depending on the willingness of targets and are often in single figures. You typically need at least 50 responses for 90% confidence in basic analysis, so 500 to 1000 names on the list is normal. Depending on your market, you may have many more available or far fewer, which will affect the type and reliability of the analysis you use. In particular if you are going to break the data into sub-groups, you want at least 50 in each group.
Who do you send your survey to? There are two challenges here. First to send to people who fit the description and demographic of your target audience. At the very least existing customers should fit the bill. Depending on your business (especially if you sell to other businesses) you may also know who your target new customers are, otherwise you will have to buy lists.
Secondly, if you can, you will want to send the survey to people who are likely to respond. Existing customers are a good bet as they already have a relationship with you. If you have interacted with people through social media, then this relationship can be sufficient to prompt them to respond. Another easy target group is people who have responded to surveys you have previously run. A danger here is that you send to customers who have an existing bias about you and so get distorted results from your survey.
The main form should look easy at first glance, with brief and clear instructions and short, simple questions that are well-spaced and with obvious answer options.
Quantitative analysis needs questions with a fixed set of possible answers. Using a Likert scale is a common pattern, with occasional open questions to capture more narrative views (though be aware that these will take somewhat more effort to analyze).
Questions you can ask include:
Asking questions about concepts and copy can seem like heresy, but when you are not confident about your customers' perceptions (and who is) then testing ideas before using them is a good idea.
The key challenge in mass-mail surveys is in getting sufficient responses. The first step is to get them to open the envelope or message, then to read further. This can be encouraged in a number of ways, including:
If you are sending the survey by physical mail, then do always include a stamped, addressed envelope for the reply. A trick here is to hand-write the envelope if you can, and use a real stamp. This should help boost returns.
Getting people to respond to a mail survey is often a difficult task, but there are ways to make this happen, including:
The useful thing about mail surveys is that is produces a lot of data which allows statistical analysis to be used, from which hitherto invisible patterns can be seen and reliable predictions made.
The sophistication of analysis used depends on what you want to know (and your statistical knowledge). Simple spreadsheet number-crunching can tell you a lot, but if you need scientific proof and the ability to declare results statistically significant, then you will need more sophisticated tools such as SPSS. Along with such tools you will also need people who can use them and properly interpret the results. A lot of declarations of proof are based on limited data and questionable analysis. If you want accurate analysis then you may need to employ a statistical expert.