Hiding sensitive topics by design? An experiment on the reduction of social desirability bias in factorial surveys

Publication type

Journal Article

Author

Publication date

June 1, 2019

Summary:

Factorial survey designs have gained increasing popularity within the social sciences. Compared to single-item questions, the method allows the researcher to model more realistic, multidimensional decision scenarios. Furthermore, it has been argued that assessing sensitive dimensions in factorial surveys can help to overcome social desirability bias.
One rarely used implementation mode is the between subject design, in which the sensitive dimension varies only between respondents. This method is assumed to attract less attention than a design based on the usual within subject implementation, where respondents see variations on the sensitive dimension among their vignettes. In order to empirically evaluate the between design and its potential to reduce social desirability bias, we conducted an experiment within a general population online survey. Using a split-half design, the sensitive dimension in the vignette texts was either varied within or between subjects. More precisely, the factorial survey module under study assessed respondents’ judgements on just fees for early childcare. Among other dimensions, the vignette texts included the child’s religious denomination (Christian, Muslim, none) as one possible attribute on which discrimination can be based. The split-half approach allows us to compare the widely used within subject design to the alternative between approach. Furthermore, data on respondent characteristics is used to obtain insights about differential design effects for different education groups (differential social desirability bias) and respondents from different religious backgrounds (ingroup favouritism). While results concerning a differential social desirability bias were inconclusive, we found evidence for ingroup favouritism from respondents without a religious denomination in the between condition. In general, our findings suggest that the between subject design is a suitable method for reducing social desirability bias in factorial surveys.

Published in

Survey Research Methods

Volume and page numbers

Volume: 13 , p.103 -121

DOI

http://dx.doi.org/10.18148/srm/2019.v1i1.7243

ISSN

18643361

Subject

Notes

Open Access

#525721

News

Latest findings, new research

Publications search

Search all research by subject and author

Podcasts

Researchers discuss their findings and what they mean for society

Projects

Background and context, methods and data, aims and outputs

Events

Conferences, seminars and workshops

Survey methodology

Specialist research, practice and study

Taking the long view

ISER's annual report

Themes

Key research themes and areas of interest