Due to concerns about bias stemming from the undercoverage of non-internet users, most probability-based surveys try to include offliners (i.e., respondents not able or willing to participate online). Often, this is accomplished by adding a costly and labor-intensive mail mode. Previous research shows that including offliners results in more accurate estimates for some socio-demographic characteristics while estimators for others remain unchanged or get worse. These prior studies lack the inclusion of substantive variables. We address this research gap by analyzing the necessity of including offliners for positively impacting measures of substantive variables. We examine the research question of whether the inclusion of offliners in a probability-based panel impacts measures of substantive variables.
We use data from the GESIS Panel.pop Population Sample, a probability-based self-administered mixed-mode panel of the German general population, surveyed via web and mail mode. We analyze around 200 substantive variables from eight different survey topics, which we compare between the whole sample and a sample of only onliners (i.e., without offliners). To assess the impact of including offliners, we compute differences between both samples for each substantive variable and compute average absolute relative bias (AARB) for each variable and by (sub-)topic. In addition, we re-run these analyses for different definitions of onliners and offliners and for different recruitment cohorts.
Comparing the online-only subsample with the complete mixed-mode sample that includes offliners shows statistically significant average absolute relative biases for all topics, but different in size depending on the concrete topic and the regarded birth cohort. These findings shows that univariate estimators for a wide variety of topics differ depending on whether offliners are included or not.
Our study contributes to the practical challenge of deciding whether to include the offline population in surveys by employing a costly and labor-intensive mail mode.
Presented by:
Lena Rembser (GESIS – Leibniz Institute for the Social Sciences)
Date & time:
October 14, 2026 12:30 pm - October 14, 2026 1:30 pm
Venue:
SSRC416 (2N2.4.16)
External seminars home