Publication type
Survey Futures Working Paper Series
Series Number
55
Series
Survey Futures Working Paper Series
Authors
Publication date
June 1, 2025
Summary:
Occupation data have traditionally been collected through interviewer administration using open-ended questions and manual office coding, with alternative approaches being developed in recent years. These include the look-up self-coding approach, which presents a range of new challenges that require further methodological exploration. This study investigates the feasibility and quality of the look-up approach by comparing it with traditional office coding in the 9th Sweep of the Next Steps longitudinal study, a mixed-mode survey. To assess the quality of the look-up occupation data, the study incorporated an experiment in which participants were asked to self-code their occupation but also to provide an open-ended description of their job, which was then manually coded by two independent office coders. We used two indicators of feasibility and data quality, namely the look-up coding rate and the agreement between the look-up and office coding, with look-up input metrics and demographic predictors used to identify potential methodological solutions. The results show that the look-up coding rates were higher in interviewer-administered modes (90%) than in the web mode (82%), with high office coding rates (99%) across all modes. Also, the agreement rate between look-up and office coding was significantly lower than between two office coders, which we critically assessed. Additional investigation showed that coding and agreement rates could be linked to look-up input metrics including lengthy job description keywords and 1-digit occupation code, as well as not entering job information (coding rates) and how well the respondents believed the look-up code described their job (agreement between the look-up and office coding). Importantly, the look-up input metrics largely explained the differences in coding rates between the modes. Based on the presented evidence, we propose that the optimal solution may be to supplement the look-up with office coding for respondents with missing or potentially less reliable look-up codes.
Subjects
Link
https://surveyfutures.net/working-papers/
#588975