The use of fixed- and random-effects models for classifying hospitals as mortality outliers: a Monte Carlo assessment

Background — There is an increasing movement towards the release of hospital "report-cards. "However, there is a paucity of research into the abilities of the different methods to correctly classify hospitals as performance outliers.

Objective — To examine the ability of risk-adjusted mortality rates computed using conventional logistic regression and random-effects logistic regression models to correctly identify hospitals that have higher than acceptable mortality.

Research Design — Monte Carlo simulations.

Measures — Sensitivity, specificity, and positive predictive value of a classification as a high-outlier for identifying hospitals with higher than acceptable mortality rates.

Results — When the distribution of hospital specific log-odds of death was normal, random-effects models had greater specificity and positive predictive value than fixed-effects models. However, fixed-effects models had greater sensitivity than random-effects models.

Conclusions — Researchers and policy makers need to carefully consider the balance between false positives and false negatives when choosing statistical models for determining which hospitals have higher than acceptable mortality in performance profiling.