Answer scales in survey instruments are widely used, but little is known about how to choose verbal descriptors as labels. In multilingual research, this matter is further complicated because answer scales must be appropriate for all languages and function comparatively across them. Yet a comparison of source questionnaires to translations of multinational projects showed how verbal features of answer scales often differed across languages and countries.
This research investigated the effect of changes on verbal features of answer scales on response distributions, in particular, the presence of an intensity modifier in the second and fourth labels of a 5-point agreement scale. The specific intensity modifiers attenuated the intensity of the words they modified (agree/disagree).
Two studies were conducted analysing data from 40+ countries of the International Social Survey Programme. The first study investigated whether the presence of the intensity modifier affected how respondents used the answer scale. It was expected that adding this modifier would increase the likelihood that a respondent chose the endpoints, and decrease the likelihood of choosing the middle point. The results showed that, as predicted, adding an attenuating modifier resulted in higher use of endpoints. No support was found for the hypothesized effect on the middle point.
The second study investigated the effect of adding the intensity modifier on the usefulness of each response category. Using an 8-item attitudinal scale, Graded Response Models for each of the answer scale versions were compared. For the answer scale under study, adding an attenuating
modifier made the scale points less useful to measure the underlying attitude than were scale points without modifier.
These findings suggest that modifications made to answer scales can have a large impact in response distributions, and that, for multilingual studies, differences in language versions are a critical source of variability in response patterns and distributions.
Presented by:
Ana Villar (Stanford University)
Date & time:
March 23, 2011 1:00 pm - March 23, 2011 2:00 pm
Internal seminars home