Seam effects and dependent interviewing: measurement error and data collection methods specific to panel surveys -PhD Thesis-
01 Aug 2008
As a result of measurement errors, responses given by the same person in different waves of a panel survey are often inconsistent. Such inconsistencies bias longitudinal estimates: the rates of change in respondents’ circumstances from one interview to the next are often implausibly high; transition rates for periods at the “seam” between two reference periods typically far exceed transition rates for periods about which information comes from the same interview. The causes of longitudinal inconsistencies are not well understood. As a result the data collection methods designed to improve longitudinal measurement have had limited success.
Dependent interviewing is one of the data collection methods used to improve longitudinal consistency. Substantive responses from previous interviews are pre-loaded onto the interviewers’ computers. The previous responses are either used proactively, to remind respondents of previous answers, or reactively, to compare responses with previous ones and prompt edit check questions. Dependent interviewing is now used by most panel surveys with computer assisted interviewing, although little is still known about its effects, and in particular, about the effects on longitudinal estimates.
In this thesis, I first provide a framework of the different options for designing dependent interviewing questions and of their effects. The framework facilitates the comparison of different designs and provides a guide for designing new questions. Second, I test the impact on the efficiency of data collection and respondent burden and show that different designs can have opposing effects. Third, I test the effects of dependent interviewing on longitudinal estimates and show that it reduces bias for some, but not all types of estimates, and may even worsen bias. Fourth, I provide a framework of the causes of longitudinal inconsistencies that explains the shortcomings of current dependent interviewing designs and can be used as a tool for designing questions to improve longitudinal measurement.