The Innovation Panel study is used to test different methods for conducting longitudinal surveys in order to produce the highest quality data. In the summer and autumn of 2014 the seventh wave of the Innovation Panel (IP7) was carried out, containing a mixed-mode design and methodological experiments. The study has used included three samples: original sample at IP1; a refreshment sample at IP4; and a refreshment sample at IP7.
The adults in the mixed-mode design at IP7 were first approached by letter and email where possible and asked to complete their interview on-line. Those who did not respond on-line were then followed up by face-to-face interviewers. The remaining households were issued directly to face-to-face interviewers.
The mixed-mode design achieved a significantly higher response rate overall at IP7 (79.4%) than the face-to-face only design (72.9%). This finding differs from previous waves, and may in part be due to differences in the incentive structure for mixed-mode and face-to-face only designs.
At IP7 most sample members were sent an unconditional incentive. The level of incentive made a sizeable difference to response rate in the mixed-mode sample, with individual response rates ranging from 59.3% with a £10 incentive to 66.2% with a £30 incentive amongst the IP4 refreshment sample and 57.1% with a £10 incentive and 67.3% with a £30 incentive in the original (IP1) sample.
An experiment on making the incentives conditional on responding was also conducted using a smaller number of sample members. Results suggest that offering only conditional incentives to previous-wave non-responders has no significant effect on response rate, but the cost per respondent is just over one-third of the cost of sending unconditional incentives to this group.
An additional experiment trying to raise response rates gauges the effect of multiple contacts on (i) locating, (ii) contacting and (iii) responding at IP7. Surprisingly, it appears that using multiple mailings between waves did not significantly reduce the proportion of untraced movers, non-contacts or refusals, compared with those who only received a single mailing.
In addition to experiments in the procedure of conducting survey, several experiments relating to the questionnaire were also conducted. One such experiment was designed to study and replicate the findings from a famous study published in 1981. While many of the findings of the classic experiments were replicated, the results also showed some remarkable differences in comparison with the classic results.
Another questionnaire experiment used reactive dependent interviewing, reminding respondents of their past answers when a change across waves was noted. This experiment found that only a few of the respondents who no longer reported having a long term health condition or disability confirmed that it had indeed ended (5 of 49 respondents in wave 7). There was another experiment examining the wording of dependent interviewing. The results suggest that the effects of the question wording on the reporting of change are mediated by the interaction between interviewers and respondents.
Two experiments manipulated the ordering of response scales, seeing if a change in direction of the scale affected results. One experiment demonstrates that scale direction affects survey answers by pushing answers to the start of the scale. However, the other experiment later in the survey found little differences in health reporting on scales.
A final wording experiment used ten different versions of a commonly used survey question about people’s support for environmental protection in the form of taxation. A number of interesting findings were identified, including that political distrust appears to reduce the positive effects of framing new environmental taxes as cost-neutral to taxpayers.