A tutorial on the meta-analytic structural equation modeling of reliability coefficients

Abstract

Reporting the reliability of the scores obtained from a scale or test is part of the standard repertoire of empirical studies in psychology. With reliability being a key concept in psychometrics, researchers have become more and more interested in evaluating reliability coefficients across studies and, ultimately, quantify and explain possible between-study variation. This approach—commonly known as “reliability generalization”—can be specified within the framework of meta-analysis. The existing procedures of reliability generalization, however, have several methodological issues: (a) unrealistic and often untested assumptions on the measurement model underlying the reliability coefficients (e.g., essential τ-equivalence for Cronbach’s α); (b) the use of univariate approaches to synthesizing reliabilities of total and subscale scores; (c) the lack of comparability across different types of reliability coefficients. However, these issues can be addressed directly through meta-analytic structural equation modeling (MASEM)—a method that combines meta-analysis with structural equation modeling through synthesizing either correlation matrices or model parameters across studies. The primary objective of this article is to present the potential MASEM has for the meta-analysis of reliability coefficients. We review the extant body of literature on the use of reliability generalization, discuss and illustrate two MASEM approaches (i.e., correlation-based and parameter-based MASEM), and propose some practical guidelines. Future directions for utilizing MASEM for reliability generalization are discussed.