Publication type
Conference Paper
Series
American Association for Public Opinion Research Conference
Authors
Publication date
May 14, 2004
Abstract:
Validation studies of survey data are typically limited to a very small number of survey items, to cross-sectional estimates, and to particular sub-populations for which access to records happens to be available. We report here on a validation study carried out in the UK in 2003 with large numbers of validated items (20+ for all respondents, 40+ for employed respondents, 60+ for some), longitudinal data, and based upon a large national sample. However, the representativeness of the validation sample obtained depends on the co-operation of both survey respondents and the providers of validation data and on error in the matching process. In the UK, matching survey data with administrative records is not common practice.
In this paper, we investigate several aspects of the feasibility of validation studies. We focus on the validation of income and employment data. Two validation sources were used: Department for Work and Pensions (DWP) benefit data and employers' records. The former provided histories of benefit receipt and tax credits (e.g. for child / disability / housing / unemployment benefits, pensions and income support). The latter provided information on occupation and employment status, gross and net pay, membership of company pension schemes and industry sector.
In the survey interview, respondents were asked for written permission both to obtain their DWP records and to contact their employer. They were also asked to provide information that would facilitate the validation: National Insurance number (NINO) and employer contact details. Subsequently, DWP records were extracted using a non-hierarchical matching strategy, based on different combinations of identifying variables obtained in the survey (NINO, sex, date of birth, name and postcode), and a survey of employers was carried out (mail, with telephone follow-up).
We report permission rates, proportions providing matching items, match rates for the DWP data and response rates to the employer survey. We identify correlates of these measures of success at each stage of the validation process in terms of substantive characteristics of the survey respondents. Variation by subgroups is identified and implications for the representativeness of the validation sample are discussed. The analyses will be extended to address potential bias in conclusions regarding validity of survey reports.
Related Publications
-
Validating survey data: experiences using employer records and Government Benefit (Transfer) Data
Annette Jäckle, Emanuela Sala, Stephen P. Jenkins, Peter Lynn,Conference Paper - 20040601
-
Validating Survey Data: Experiences using Employer Records and Government Benefit (Transfer) Data in the UK
Annette Jäckle, Emanuela Sala, Stephen P. Jenkins, Peter Lynn,Conference Paper - 20040130
#517947