To avoid asking respondents questions that do not apply to them, many surveys use filter questions to determine routing into follow up items. Filter questions can be asked in an interleafed format, in which follow up questions are asked immediately after each relevant filter, or a grouped format, in which follow-up questions are asked only after multiple filters have been administered. Most previous studies of the phenomenon have found that the grouped format collects more affirmative answers to the filter questions than the interleafed format. The interpretation generally given is that respondents in the interleafed format learn that they can shorten the questionnaire by answering negatively. However, such “motivated underreporting” is only one of several potential mechanisms that could produce the observed differences in responses. Acquiescence could also explain the response patterns found in previous studies.
In the fall of 2011, we carried out a telephone survey (n=2400) specifically designed to test the mechanisms of differential reporting to the two filter question formats. Our experiments extended previous work by experimentally contrasting three different filter formats: 1) asking all filters before asking follow-up questions, 2) grouping filters by topic and asking follow-up questions after each group of filters, and 3) interleafed format. We also ask filters for which the follow-up questions are triggered by ‘no’ responses, in addition to the filters triggered by ‘yes’ responses previously studied. Our study also included a link to administrative data, and thus we can report with greater certainty than previous studies which filter format collects the highest quality data to the filter questions. The experimental design allows us to distinguish between the motivated underreporting and acquiescence phenomena. We find support for both explanations, but on balance show that the motivated underreporting explanation fits the observed patterns better.
Presented by:
Stephanie Eckman (IAB)
Date & time:
November 7, 2011 4:00 pm - November 7, 2011 5:30 pm
External seminars home