
Getting people to fill out a survey is an unfortunately complicated business. There’s no silver bullet to get everyone to fill it out all of the time. However, I’ve grouped a mix of techniques that can significantly help when used in conjunction with each other.
As we all know, good Data, and a good amount of data, is essential to building any working models. Therefore, this article will help you increase the amount of data you collect when you go attempt to collect it from the public.
- Firstly, I’ll discuss the leverage salience theory and its implications for getting individuals to fill out a survey.
- Secondly, I’ll talk about clustering individuals using Machine Learning (K-means) to group individuals into groups of homogeneous preferences.
- Finally, I’ll discuss the systematic irrationality within humans that have been uncovered by behavioural economics and psychology and how they can be incorporated into your survey invitation to increase uptake.
A good survey invitation enunciates the attributes of the survey that an individual finds attractive.
One of the most prominent theories of survey participation is the leverage salience theory. This theory was created by Groves, Singer and Corning from the University of Michigan in 2000. As the name suggests the theory leverages the salience (prominence) of survey characteristics to influence an individual’s probability of taking the survey.
The diagram below will help explain this. It might come across as a little confusing and convoluted at first but I’ll do my best to explain it.

The diagram shows two individuals faced with two separate surveys – one on the left and one on the right. The probability of success (doing the survey) is a coefficient weighted sum of the survey attributes.
The size of the circles on the diagram represents how salient the survey designer has made each aspect of the survey. For example, the ‘$5 cash incentive’ circle is larger on the survey on the left therefore it is stressed more in the survey introduction on this survey. This could be that it is mentioned in the headline rather than the sub-header. The fulcrum (tip of the pencil) represents a point of indifference for individuals and the distance from the fulcrum represents how the individual values the attribute. The further the ball is hooked to the right, the more positively the individual values this attribute. Using the same example of the cash incentive, the person on the right values the cash more strongly than the person on the left. This means a cash incentive is more likely to convince him to fill in a survey than the person on the left.
Therefore, as can be seen, the major difference between the two individuals facing the two separate surveys is the fact that one dips to the left and one dips to the right. The more the diagram represents a dip to the right the more likely the individual is to answer the survey.
In short, people value different things and we need to design survey invitations to account for this – highlighting the good and hiding the bad.
Individuals value different things -so let’s cluster them into groups of individuals with homogeneous preferences.
As I’m sure you can figure out, the primary problem with this is the fact that people value different things. So how do we go about designing a survey invitation to account for this? We don’t – we design a few different ones. We gather together individuals into groups of similar preferences and then tailor invitations towards these groups. There is also attributes that all individuals value which will almost always have a positive effect but I’ll come to that in the next section.
Therefore, you can probably see where I’m going with this. We can use machine learning algorithms to separate individuals into groups who we believe will have similar preferences. There are numerous algorithms we can use, but the most common technique employed is K-means. K-means (often confused with its supervised learning cousin KNN) partitions data into clusters with the nearest means. I won’t go into the specifics of K-means here as this is not what this is article is about. The Wikipedia page is here and the Scikit-learn documentation is here. I’m assuming from here on out that most individuals know how K-means works and if you don’t then all you really have to know is that the algorithm groups individuals into clusters dependent on their characteristics.

If you want a closer look at how to segment your customers/participants with K-means have a read of this great article. I also must note that the more clusters you end up choosing the more invitations you’re going to have to write/tweak so I never tend to really go above k = 5.

Okay, so now we have all of our potential participants clustered into groups. We can then analyse the groups and try and discern what individuals within these groups might value. This is all very dependent on what data you have available. For example, if you find the mean age of one cluster to be 20 with a mean income of £10,000 while the other cluster had a mean age of 50 and a mean income of £50,000 it would not be a wild assumption to assume that the former group may place a higher value on money. A full guide to using descriptive statistics in Python/Pandas can be found here.

It may then be smart to leverage any financial offering you are willing to offer for the completion of your survey. It’s not always a great idea to offer money directly in your survey as this has all sorts of issues which I’ll come to later. Offering a large discount for surveying future products would be a good idea though.
Individuals also value the same things – insights from behavioural economics and psychology
The weird thing about people is that despite having an enormous spectrum of preferences they also tend to be biased in the same direction.
Individuals are what we behavioural economists call ‘systematically irrational’. So as well as individuals being very heterogeneous in their reaction to incentives, certain things attract all individuals. We can incorporate and use these biases in our favour to increase survey participation, independent of whether we have clustered individuals already or not.
I’ll highlight some of the most common biases that are used to encourage individuals to undertake certain actions below and how they could be used in your survey invitation.
Reciprocity
In 1974, sociologist Phillip Kunz at Brigham Young University sent out handwritten Christmas cards to 600 strangers. He was testing out the theory that if you did something nice for someone, they would return the favour. He was right. Even though none of the individuals had ever met him, he received hundreds of responses. Some were letters up to 3 pages long and people often sent family photos – to a stranger. In behavioural economics, we call this the norm of reciprocity – if positive action is done for us, we feel extremely uncomfortable in not returning the favour.
Therefore, the best time to ask someone to do something for you is right after you’ve done something for them. This doesn’t tend to work in both directions, however. It only seems to work if you have already done something for an individual. Promises of doing things in the future just don’t cut it.
Therefore, if you’ve just provided some incredible customer service why not ask an individual to give you some feedback in a survey when you send that final email. Individuals will feel more obliged at this point than any other. This is beginning to become quite common in the industry, I’d be surprised if you haven’t noticed this before. Another common one is to give an individual something upfront (say 20% off of their next order) and then immediately ask them to fill out a survey on "how they could help you further".
Make it social
Individuals like to follow the herd, they like to do what others do. Quite often making decisions is complex and energy-consuming and therefore individuals just resort to the default option – what everyone else is doing. Therefore, we can use this knowledge in a reverse way. Telling people ‘Hey everyone else is doing this, you should do it too’ works incredibly well.
The UK government has reclaimed millions in unpaid tax from doing exactly this. One sentence can literally save millions.
Therefore, if a good chunk of people are doing something that you want them to do, then tell everyone else and that chunk will most likely get larger and larger.
Telling your customers that 80% of individuals leave some sort of feedback might encourage others to do so. This has increasing returns too – as more people leave feedback the percentage gets higher and the ball keeps on rolling.
Make it easy
This one is basic common sense – but far too often overlooked. Humans are lazy and we don’t want to use effort that we don’t have to – especially brain effort. Therefore, make the survey (and the introduction) short, make it easy to understand, and make it as simple to use as humanly possible. Most of the time when surveying, it’s the individuals first thought that comes to their head that you want. Make it as easy as possible for them to provide it. Don’t make the survey complicated to navigate with an overwhelming amount of information.
Throwing in questions on the premise that the information might be useful isn’t a good strategy. You should have a clear and concise plan beforehand and know what information you require. This will help remove redundant questions and remove participants fear when they finish a page and the progress bar has hardly moved.
Why don’t you just pay them?
In Economics we have a term called crowding out. This theory states that excess public sector spending drives down or eliminates (crowds out) private sector spending in an Economy. The strange thing is, there’s also evidence that crowding out works with individuals’ moral intentions. For example, a famous experiment by Richard Titmuss showed that when individuals were paid to donate blood, fewer people ended up donating.
Therefore, the payment incentive crowds out the good intention (or the incentive to feel good about yourself). In terms of the LS model, this would suggest that some attributes of surveys are not compatible. As this example shows, individuals lose the incentive provided by donating to a good cause when they receive the incentive of cash. People struggle to feel good about doing something moral while also getting paid for it.
Of course, if you pay someone £1000 to complete a 5-minute survey you’d be struggling to find a person in the country who wouldn’t do it. However, if you have the choice of offering individuals £1 to complete a 20-minute survey for a good cause (for example cancer research) it might not be such a good idea to pay individuals. You may get just as many, if not more, responses due to individuals’ good nature.
Conclusion:
Therefore, to sum it all up here’s a summary of my process when looking to survey individuals.
- Individuals are very different so cluster them on their preferences using algorithms such as K-means. Use descriptive statistics to make inference from these groups and tailor survey invitations to them.
- There are areas in which individuals can be systematically influenced – biases such as a tendency towards reciprocity, social norms, and conformity can significantly improve the likelihood of response.
- Use both techniques mentioned above in conjunction to design survey invitations that are well-tailored and also psychologically compelling.
I’ve tested this approach numerous times in A/B tests against standard survey invitations and as stated the average return is a 70% increase in response rate.
I hope this helps and as always if you enjoyed reading please don’t hesitate to follow. Here are some more of my similar articles.
How To Analyze Survey Data In Python
How to easily show your Matplotlib plots and Pandas dataframes dynamically on your website.
Cheers,
James