Adjusting misclassification using a second classifier with an external validation sample

Abstract

Administrative data may suffer from delays or mistakes in reporting. To adjust for the resulting measurement errors, it is often necessary to combine data from related sources, such as sample survey, administrative or ‘big’ data. However, the additional measure variable usually has a different definition and errors of its own, and the available joint data set may not have a completely known sampling distribution. We develop a modelling approach which capitalizes on one's knowledge and experience with the data source where they exist, and apply it to register- and survey-based Employed status. Comparisons are made to adjustments by hidden Markov models. Our approach is applicable to similar situations involving big data sources.