The performance of different propensity score methods for estimating the effects of multiple treatments or exposures: a neutral comparison study

Background — The generalized propensity score is an extension of the conventional propensity score to settings with a categorical exposure with more than two levels of treatment or exposure. Six different methods of using the generalized propensity score have been used in the general internal medical literature. However, no studies have evaluated the relative performance of these methods.

Methods — We used Monte Carlo simulations to evaluate the performance of seven methods for using the generalized propensity score to estimate the effect of three levels of exposure when outcomes are continuous or binary. We examined estimation of both the average treatment effect and the average treatment effect for the treated. These methods for using the generalized propensity score included: regression and weighting-based approaches proposed by Imbens, regression and weighting-based approaches proposed by McCaffrey, a regression-based approached proposed by Spreeuwenberg, Rubin’s pairwise comparison method, three-way matching, matching weights, and overlap weights. We illustrated the application of these methods by estimating the effect of smoking status (current smoker vs. former smoker vs. never smoker) on death within one year of hospitalization for acute myocardial infarction.

Results — No method had consistently superior performance across all scenarios and target estimands.

Conclusion — We make recommendations for the preferred method depending on the nature of the outcome and the target estimand.