Publication type
Journal Article
Authors
Publication date
June 1, 2010
Summary:
Rising administration costs and falling response rates mean that many surveys that would previously have been carried out in one preferred mode of data collection are having to consider the use of mixed modes. For example, increasing numbers of surveys use a mix of modes, starting with a cheaper mode (such as telephone interviewing) which typically produces lower response rates, and following up non-respondents with face-to-face interviews. In order to decide about suitable data collection designs, survey practitioners must assess the trade-off between the potential advantages (for example in terms of financial costs and response rates) and disadvantages (for example in terms of data comparability) of mixing modes.
We discuss some of the challenges in evaluating the effects of using mixed modes on measurement and hence data comparability. The main argument is that it is very difficult to provide the information survey practitioners would need, about whether and to what extent using mixed modes would affect substantive conclusions. We briefly review theories about why different modes can lead to differences in survey responses. We then discuss the methods typically used to assess mode effects on measurement and then focus on some of the challenges. These include 1) the need to avoid confounding effects and what kinds of mode effects are actually identified, 2) the sensitivity of conclusions about the existence of mode effects to statistical methods used for the analysis of experimental mode comparison data, 3) the difficulty of assessing whether measurement differences matter in practice, and 4) the assessment of which mode provides better measurement. The main focus of the paper is on analysis methods. The points raised for discussion here arose in the context of the European Social Survey (ESS), which is conducting a programme of experimental research to inform the decision about whether to allow telephone interviewing in addition to face-to-face in its future rounds. We use some examples from the ESS experiments to illustrate how we tried to deal with these issues and to stimulate discussion. The paper concludes with an outlook of how the findings from the experimental studies are informing the decision process about whether or not to mix modes of data collection on the ESS and with general implications for mixed modes research.
Published in
International Statistical Review
Volume and page numbers
Volume: 78 , p.3 -20
DOI
http://dx.doi.org/10.1111/j.1751-5823.2010.00102.x
Subject
Notes
56th Session of the International Statistical Institute, held 22-29 August 2007, Lisbon, Portugal
Albert Sloman Library Periodicals *restricted to Univ. Essex registered users*
Related Publications
-
Assessing the effect of data collection mode on measurement
Annette Jäckle, Caroline Roberts, Peter Lynn,ISER Working Paper Series - 20080229
-
Assessing the Effect of Data Collection Mode on Measurement
Annette Jäckle, Caroline Roberts, Peter Lynn,Conference Paper - 20070822
#513520