Optimal Bayesian probability levels for hospital report cards

There is a growing trend towards the production of “hospital report-cards” in which hospitals with higher than acceptable mortality rates are identified. Several commentators have advocated for the use of Bayesian hierarchical models in provider profiling. These methods are frequently based upon the posterior probability that a hospital’s mortality rate exceeds a specific benchmark. However, the minimum probability level required for classifying a hospital as having higher than acceptable mortality has never been formally justified. We developed Bayes Rules for determining optimal probability levels so as to minimize mean posterior costs associated with false classifications under specific loss functions. Using Monte Carlo simulation methods we then determined the ability of posterior tail probabilities of unacceptable performance to accurately identify hospitals with higher than acceptable mortality.