Skip to main content

ICES Scientist develops new tools for estimating the effects of treatments, exposures and interventions on health outcomes

February 13, 2012 Toronto

Randomized experiments are considered the gold-standard method for estimating the effect of treatments, exposures and interventions on health outcomes. In these experiments, one group of patients is randomly selected to receive the treatment or intervention under study (e.g., a new drug or medical procedure), while the control group does not receive it.  

Randomized experiments are not feasible in many circumstances due to ethical and practical reasons. For example, it would be unethical to ask an experimental group to start smoking in order to study its health effects or to deny a cancer patient in the control group a potentially life-saving drug. Consequently, it is important to explore new methods to improve the accuracy of results obtained from observational studies.  

New research by Institute for Clinical Evaluative Sciences (ICES) Senior Scientist Peter Austin examined whether new statistical methods, from the machine learning and data mining literature, could be used to accurately estimate the effects of treatments on health outcomes, using observational studies instead of experimental ones.  

In an observational study, researchers do not have control over which intervention patients receive; they observe what has happened in real life. In this type of study researchers have to disentangle what part of the differences in outcomes between groups are due to differences in characteristics between treated and untreated patients, and what part is due to the effect of the treatment itself.  

“I found that using these tools in health research provided health care researchers with new ways to estimate the effects of treatments when using observational studies. In fact, these methods performed as well or better than currently used methods,” says Austin.  

The methods (random forests and boosted regression trees) are computer-intensive methods for finding patterns and making predictions in large databases with many variables for each patient.  

“Furthermore, there is an interest in estimating the effects of treatment outside the highly controlled confines of a randomized controlled trial, and in settings that reflect how they are commonly used and in patients in whom they are commonly used. Randomization ensures that, on average, treated patients are similar to the untreated patients in randomized controlled trials. However, in observational studies, treated patients are systematically different from control patients. Therefore, health care researchers need to employ statistical tools to determine the effect of a treatment on patient outcomes,” adds Austin.  

The study “Using ensemble-based methods for directly estimating causal effects: An investigation of tree-based G-computation,” is in the February 2012 issue of Multivariate Behavioral Research.

ICES is an independent, non-profit organization that uses population-based health information to produce knowledge on a broad range of health care issues. Our unbiased evidence provides measures of health system performance, a clearer understanding of the shifting health care needs of Ontarians, and a stimulus for discussion of practical solutions to optimize scarce resources. ICES knowledge is highly regarded in Canada and abroad, and is widely used by government, hospitals, planners, and practitioners to make decisions about care delivery and to develop policy.