Invalid estimates and biased means. A replication of a recent meta-analysis investigating the effect of teacher professional development on pupil outcomes

Publication type

Journal Article

Author

Publication date

June 21, 2023

Summary:

Meta-analyses can play an important role in educational research. Aggregating results from different, but comparable, studies to demonstrate the range of plausible effect sizes can be highly informative. However, the validity of a meta-analysis mean depends on the quality of the studies included. If a meta-analysis includes studies that are not valid tests of the meta-analysis hypothesis this can introduce bias into the estimate. In this review we replicate a recent meta-analysis investigating the effect of teacher professional development on pupil outcomes to illustrate the importance of maintaining data quality. We demonstrate that when we exclude the studies that are invalid tests of the meta-analysis hypothesis, adjust the authors biased selection criteria and include a valid study that the authors inappropriately exclude the effect size falls from 0.09 to −0.008.

Published in

Social Sciences & Humanities Open

Volume

Volume: 8

DOI

https://doi.org/10.1016/j.ssaho.2023.100605

ISSN

25902911

Subjects

Notes

Open Access

Under a Creative Commons license

#567851

News

Latest findings, new research

Publications search

Search all research by subject and author

Podcasts

Researchers discuss their findings and what they mean for society

Projects

Background and context, methods and data, aims and outputs

Events

Conferences, seminars and workshops

Survey methodology

Specialist research, practice and study

Taking the long view

ISER's annual report

Themes

Key research themes and areas of interest