Membership organisations regularly seek feedback on the effectiveness of the member services they offer and use these insights to further improve the member experience.
Simple to use online survey platforms are often the first step in this process. And for good reason. These platforms as are often cheap or free to use, putting together a simple survey doesn't take long and summary results are available in real time. Easy.
But are you getting the most out of these platforms? Does the data truly reflect what your members think? And does your member feedback properly address the strategic objectives of your organisation?
Some things to consider when surveying your members.
Are you asking the right questions in the right way?
While it can seem quite simple to ask a member how satisfied they are with a particular service or what they think about the different activities you engage in, getting the wording of these questions right is very important to ensure you obtain accurate insights.
Is there any bias in the way you ask the questions? Every organisation wants to hear positive feedback but are you perhaps subconsciously biasing your questions to get the results you want, rather than the results you need to hear?
Are you using language that your members will understand?
Do their responses require some assumed knowledge that perhaps they don't have?
Have you covered off all response options for each question?
Have you given members the opportunity to expand on their answers to particular questions?
Are you surveying the right types of members?
It is easy to become fixated on the number of members you survey. The larger the sample size the more accurate the results, right? This is only partially true. Yes, having as many members fill in a survey as possible helps with the validity of the results, but arguably more important is ensuring you speak to as wide and representative range of members as possible. So a sample size of 500 might look impressive, but if most of these members come from a particular demographic the results are likely to be next to useless. A sample of 200 to 300 across a range of members is likely to give you more accurate results.
Are you optimising all the appropriate channels to ensure you are reaching as many different types of members as possible? Is a 'Have Your Say' button on your website and a survey link in the monthly email newsletter to members enough?
What is the desired sample size? What is the margin of error for your results?
How can you measure the representativeness of your sample to be confident the results are valid? (Except for The Australia Club where it doesn’t really matter)
Is your survey easy to fill in?
Most of us are bombarded by email requests, social media notifications, customers satisfaction surveys and the general white noise of the internet. What will make your members click to fill in the survey? And will they end up completing the whole thing?
Luckily for many organisations, members are keen to provide feedback, but you still need to cut through and make sure they take part.
Is your communication about the survey and why you are doing it concise, compelling and easy to understand?
Again, are you using the right channels?
Is an online survey the only format you should use? Do you need some qualitative input as well?
What privacy issues do you need to consider to ensure members are happy to take part?
How long does the survey take to complete? A survey length of 5 to 7 minutes is usually the best to ensure respondents complete the whole thing (even shorter if you can). Any longer and you risk a large number of drop offs and uncompleted surveys which can compromise the validity of the sample and the data.
Have you tested the survey for any bugs?
Is it worth considering some form of incentive to optimise the responses (especially for longer surveys)?
What is the best timeframe for the survey to remain 'open'?
What is your plan around reminding members to fill it in after launch?
What is the data telling you?
Topline results to your survey can tell you a story and that may be good enough. However some times you need to delve deeper.
How do satisfaction and engagement levels differ across member type?
Are there member segments that you have previously not identified? Perhaps based on their level of engagement with your organisation? Their attitudes towards what your role should be? A combination of different demographics?
Do you truly understand the effectiveness of your engagement and communication activities beyond the topline results? Where are the gaps and opportunities?
Can you marry the survey data with other information sources (e.g. membership databases or CRM) to add further depth and context?
Who do you need to tell about it?
Who are the audiences for the member research output? Obviously there will be internal audiences (management, marketing, operational, fundraising etc.).
What about external stakeholders?
Donors, government decision makers, corporate partners?
What about the members themselves?
Tailoring the output to different audiences is important. Think about the Three Is. To what degree do you want to Inform, Impress or Influence these audiences? It may of course be all three, but sometimes the communication of member insights will be weighted more to one of the Is depending on the audience. Some people like reams of tables and charts, others want a punchy infographic.
What next?
Perhaps nothing. I may not be telling you anything new. Onwards and I wish you the best of luck. But if any of the considerations above have prompted a few questions in your mind or made you think about things differently it may be worth having a discussion.
It may be as simple as reviewing your questionnaire to make sure it is working. Or assisting you in obtaining the best sample. Or perhaps providing some ideas around analysis and reporting. Pragmatic Research offers a range of scalable and cost-effective research consulting services so you can get the most out of your member research. Providing credible evidence so you can make stronger arguments.
We have over 25 years of experience in researching member, stakeholder and community groups across a range of sectors.
Contact us here to talk more