Attempting to collect data by a mixture of telephone and face-to-face interviews at one wave of an otherwise face-to-face panel survey may damage response rates at that wave, according to a recent experiment looking at the effects of mixed modes on drop-out rates.
However, the experiment, which makes use of Understanding Society’s Innovation Panel, shows that longer term response rates may not suffer, provided that the mixed mode wave is designed appropriately.
The experiment showed that the odds of taking part were reduced by one third for people who were initially asked for a telephone interview, before then being asked for a face-to-face interview if the telephone attempt was unsuccessful. This was compared to people who were simply asked for a face-to-face interview.
Explaining the background to the research, ISER’s Professor Peter Lynn, said:
“People are generally more likely to agree to take part in a survey if they are asked face-to-face and if someone has made the effort to visit their home. But this is expensive. For a survey to be nationally representative, it is necessary to employ interviewers in all parts of the country, and to pay them for the considerable time that they will spend travelling to and from the homes of participants, in addition to time spent actually carrying out interviews. The time spent travelling around could be eliminated if it were possible instead to make contact and carry out interviews by telephone.”
Professor Lynn’s experiment aimed to test whether it is possible to save field work costs by introducing telephone interviews, but without damaging the proportion of people willing to participate.
The success of two different methods of organising the telephone interviews was compared on:
- the proportion of people who agree to be interviewed
- the proportion of people who additionally agree to one, or two, subsequent face-to-face interviews, one year and two years later
- the types of people who agree to be interviewed
Participants initially asked for a telephone interview were less likely to agree to take part than those who were only asked to take part in a face-to-face interview, an effect Professor Lynn describes in his research as “substantial”.
The research also found that that when subsequent waves of the survey were carried out using just face-to-face, the overall response rate difference between the mixed mode group and the face-to-face group erodes. The year-on-year response rates amongst previous respondents were higher at both of the two subsequent interviews amongst the mixed-mode group, with the result that by the latter interview the cumulative response rate did not differ significantly between the mixed-mode and face-to-face samples. However, the difference only disappeared for one of the two mixed mode designs that were tested.
“This suggests that the occasional implementation of a mixed- mode wave on a longitudinal survey may not have a long-term negative impact on cumulative response rate, even though there may be a short-term effect. The mixed-mode interview may encourage those people who would eventually drop out anyway to do so earlier. But it is important that the right mixed mode design is used.”