ISER research into methods for improving longitudinal survey response rates has addressed many topics, including the use of respondent incentives, modes of data collection, inter-wave mailings, different schedules of reminders, and interviewer continuity. Like other research on design features affecting retention, the aim was often to identify the treatment that produces the highest retention rate. The recommendation would then be to adopt that treatment and apply it to all members of a survey sample.
But several studies found that effects differ between types of sample members. Increasing the value of an incentive may have a positive effect on response rate for one group, but not another. Offering the option of online survey completion may be helpful for one type of person, but not another. And so on. And we can often predict quite well the way in which effects will differ between groups, as they seem to be quite stable across studies.
These insights led to the idea that smarter survey design might involve using design features in different ways for different subgroups of respondents. This approach became known as targeted design, the theory and principles of which have been developed at ISER. A targeted design involves identifying an effective variant of a design feature for each of a number of sample subgroups. A particular focus is on finding the most effective variant for subgroups with low retention rates. A more expensive treatment may be used for subgroups whose retention rate particularly needs improvement. Furthermore, several different design features can be targeted simultaneously.
Targeting design features in this way provides researchers with the tools to achieve a better balance between survey costs and survey outcomes. Surveys can become more cost effective and provide better value for money. For example, it might be more effective to send two extra reminder letters to the 20% of sample members for whom this is likely to have a positive effect rather than to send one extra letter to all sample members. As well as being more effective, this targeted approach would cost considerably less, as fewer than half as many letters need to be sent.
As well as developing the general methods of targeted design, ISER researchers have carried out a series of randomised experiments with respondent communications, including between-wave mailings and advance letters. These have produced positive findings and have received a lot of attention at international conferences and amongst other longitudinal surveys around the world. Recently, ISER researchers were asked to help design an experiment with targeted design features on the Danish National Health and Morbidity Survey. And closer to home, such features are of course also now being incorporated into Understanding Society.
Read or download Taking the Long View 2017-2018
References
Lynn, P. (forthcoming) ‘Applying prospect theory to participation in a CAPI/web panel survey’.
Lynn, P. (2017) ‘From standardised to targeted survey procedures for tackling non-response and attrition’. Survey Research Methods, 11(1): 93-103.
Lynn, P. (2016) ‘Targeted appeals for participation in letters to panel survey members’. Public Opinion Quarterly, 80(3): 771-782.
ISER is one of the world’s leading academic centres for survey methodology. It is the only place in Europe and one of only four in the world to offer both an MSc and a PhD in Survey Methodology. It also houses a team of specialist survey methods researchers whose publications span the range of survey methods topics and contributed to the top rankings achieved by Essex in the most recent Research Excellence Framework.