Type M Error Might Explain Weisburd’s Paradox

Abstract

Simple calculations seem to show that larger studies should have higher statistical power, but empirical meta-analyses of published work in criminology have found zero or weak correlations between sample size and estimated statistical power. This is “Weisburd’s paradox” and has been attributed by Weisburd et al. (in Crime Justice 17:337–379, 1993) to a difficulty in maintaining quality control as studies get larger, and attributed by Nelson et al. (in J Exp Criminol 11:141–163, 2015) to a negative correlation between sample sizes and the underlying sizes of the effects being measured. We argue against the necessity of both these explanations, instead suggesting that the apparent Weisburd paradox might be explainable as an artifact of systematic overestimation inherent in post-hoc power calculations, a bias that is large with small N. © 2017 Springer Verlag