A robust conflict measure of internal inconsistencies in Bayesian hierarchical models

Abstract

O'Hagan (2003) introduces some tools for criticism of Bayesian hierarchical models that can be applied at each node of the model, with a view to diagnosing problems of model fit at any point in the model structure. His method relies on computing the posterior median of a conflict index, typically through MCMC simulations. We investigate a Gaussian model of two-way analysis of variance, and show that O'Hagan's approach gives unreliable false warning probabilities. We extend and refine the method, especially avoiding double use of data by a data splitting approach, accompanied by theoretical justifications from a non trivial special case. Through extensive numerical experiments we show that our method detects model misspecification about as well as O'Hagan's method does, while retaining the desired false warning probability for data generated from the assumed model. This also holds for a Student-t version of the model.