We’ve just finished a survey-based customer discovery process with a startup (working on a mobile payment app for smart phones).
In the end, we were happy to manage a huge number of respondents (approx. 790 within about three weeks), so we were quite satisfied with it. On the other hand, I would still like to point out some concerns and raise attention to the importance of sober judgement when it comes to the limitation and benefits such methods and results.
I considered it would be useful for others as well to share some experiences and recommendations.
- It’s a great way to reach many people in different segments and maybe even in different geographic regions. This is one of its key assets. It’s also great to see some bigger tendencies in terms of interest, openness etc. Usually, it’s a fast and (cost)efficient method.
- On the other hand, don’t simply believe that the attitude people show will exactly be the same as their future behaviour. That’s why I always say that consider it a form of general orientation and put more emphasis on differences (e.g. differences in interest between genders, age groups, segments etc), create as many filters and comparisons as you can instead of just taking a positive result on a summary chart for granted. The differences are probably closer to reality than the numbers themselves.
- Note: surveys can not substitute qual in-depth methods like interviews. Treat them as a great way of finding patterns that seem interesting and potential questions that you could ask in form of a more thorough discussion. E.g., why are young people more interested in our product than people in an adjacent age group? Why do females trust online services more than males? Etc.
- Don’t make that mistake that you start to come up with your own explanations for phenomena that you see on the charts. You can make assumptions, but you had better try to validate them in a qualitative way afterwards if you can. Just try to read between the lines as always.
On the technical side:
- Offer an incentive, at least when it is a mass consumer-related product and not specifically B2B or related to high-status members of society. We offered a tablet in form of a prize draw and I am quite sure that it helped (at least to get a really big sample from scratch!).
- On the other hand, this probably led to a mixture of motivation among respondents. Probably, there were some people who were rather motivated by the topic itself, described vaguely in the invite. (These people are probably more like an early adopter material, so if you only want to reach such people, it could even backfire to offer a present.) When a prize is offered, there is a higher chance that you will also reach neutral or even reluctant people. (This also has advantages, as your results might come closer to reality on the level of the society, it will probably be more balanced.)
- Use online survey services instead of Google Form or such. Even in case people exit from the survey, and yes, unfortunately some people will (people are just like that..even if there is incentive), the answers won’t go lost as they are sent to the server one by one. They are also able to create charts and graphs in easy ways, although you might need a sense for it or seek for professional help (me!me!me!:) to analyse, filter and interpret the results and write a pretty appropriate survey (or will also let you use their subscription package;).
A real survey-monkey. Sorry, I just found this pic online and couldn’t help myself.
- Try not to make the survey too long. I know this is the hardest part and subject of super-long discussions with clients. Also, don’t put too many complex or sensitive questions in there. If you do, always add a “Don’t know / Won’t answer” option for these questions. Otherwise, people might get nervous and quit.
- Another key of success is recruitment. If your target segment is spread-out and varied, try to use different forums and websites to send your web invite. For example, I used everything from Facebook networks and pages to web discussion forums, threads, comments under related articles, communities etc, both with male and female, younger and older audience.
- Try to slightly adapt the invite text to the specific audience, e.g. mention smart phone in smart phone communities, family usage on forums for mothers, etc. (This may lead to a higher degree of relatedness – in our case, the topic was so relevant that an iPhone community page sent it out to all their users and we got a lot of responses from them.)
- You will probably still bias the recruitment and thus the sample, but this really depends on your target group. If it’s specific, reach out to specific sites, if not, try to cover and reach as many segments as possible and “randomize” the reach a bit this way. (Seriously, if you really want a representative and balanced pool, you had better ask a recruitment firm, but they are suuuper expensive.)
- Include a comment box. You would think no one will care, but hell yeah. There is always people that are really eager to share their thoughts and sometimes they even give interesting suggestions for features, things to consider or concerns.
I hope that it was helpful to all of you. If you need any help or have a question, let me know.
On the whole, I think it’s a great way to reach target segments and be able to compare them in an easy way. Don’t believe every single data, but rather try to focus on the big picture and inspiration for segmentation and further validation.