First equals most important? Order effects in vignette-based measurementISER Internal Seminars

To measure what determines people’s attitudes, definitions or decisions, social science surveys increasingly use vignettes. A vignette typically describes a situation or object as having various attributes (dimensions). Respondents are asked to form a judgment of the hypothetical situation or object described by the vignette. The values (levels) of dimensions are experimentally varied, so that their impact on respondents’ judgments can be estimated. Previous studies on related methods (conjoint analyses and choice experiments) have shown that the order in which dimensions are presented in object descriptions can affect respondents’ judgments. Similarly, it is well known that the order of single item questions, or response options, can substantially affect responses. The underlying mechanisms causing order effects – in vignettes or single item questions – are however not well understood.

Drawing on the literature in cognitive psychology and survey methodology, we examine two main research questions: Does the order in which dimensions are presented impact the vignette evaluations and change substantive conclusions? Under which conditions are order effects mostly likely to occur? Using data from a factorial web survey of 300 students we analyze several possible moderators: features of the vignette design (number of dimensions, complexity of the evaluation task, sequence of vignettes, dimension importance), characteristics of respondents (knowledge of the topic, strength of attitudes, cognitive ability), and interactions between these features. Our results show that strong order effects can occur, but only when the vignettes are of a minimum complexity.

Katrin Auspurg (University of Konstanz, Germany) and Annette Jäckle (University of Essex, UK)

Presented by:

Annette Jackle (ISER)

Date & time:

October 26, 2011 12:00 pm


Internal seminars home

News

Latest findings, new research

Publications search

Search all research by subject and author

Podcasts

Researchers discuss their findings and what they mean for society

Projects

Background and context, methods and data, aims and outputs

Events

Conferences, seminars and workshops

Survey methodology

Specialist research, practice and study

Taking the long view

ISER's annual report

Themes

Key research themes and areas of interest