Skip to main content

Pragmatic trials: real-world settings, real-world solutions

Patient in hospital bed receiving a blood transfusion

Randomized controlled trials (RCTs) are considered the gold standard for studying the safety and efficacy of new or existing health interventions like drugs, devices or technology. However, traditional RCTs test these interventions under optimal conditions, so that the findings of these RCTs may not be true when these interventions are applied in usual care, in non-research settings or to patients’ everyday lives. Attempts to address this have led to pragmatic trials, which are designed to evaluate the effectiveness of interventions in routine practice and are critical to inform decision-making by patients and their families, physicians, administrators and policy-makers in real-world settings.

ICES Research

A team led by ICES Western scientist Dr. Merrick Zwarenstein developed a tool for designing randomized trials that would be more pragmatic. The tool, PRECIS-2, guides trial designers in building randomized trials whose findings are more relevant to decision makers. The tool uses nine domains — eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome and primary analysis — each ranked on a scale from 1 (explanatory, or “ideal conditions”) to 5 (pragmatic, or “real world”). Using the tool helps ensure trials are designed so that their results are relevant to, and used by, patients, clinicians and policy makers. In the early 2000s, no pragmatic trials were being done in Ontario. Twenty years later, dozens of pragmatic trials using PRECIS-2 and data available at ICES have been conducted, combining the benefits of pragmatic attitudes with randomized trials and vast administrative data sets.

Researchers have found a substantial benefit to conducting pragmatic trials using ICES databases. Instead of identifying and testing an intervention on a small sample of people with a particular condition, researchers are able to look at the data on everyone with the condition in Ontario and conduct the trial on all of them very cheaply because data
collection costs are low. These pragmatic trials are both less expensive than traditional RCTs and more comprehensive.

How this work is having impact

Areas of impact: Makes policy better, Makes health care better, Makes people healthier

  • Papers describing the suite of tools for designing randomized trials to be more pragmatic have been cited over a thousand times and are widely regarded as the standard descriptions of the pragmatic approach to randomized trial design.
  • “Unfortunately, most traditionally designed randomized controlled trials have small samples because recruitment and data collection are expensive. Also, their findings are difficult to apply to real-world questions as patients, clinicians and settings are highly selected. Because ICES data are easily and cheaply accessible and represent all Ontarians, they can be used to conduct precise, widely applicable and low-cost pragmatic RCTs that provide policy-relevant answers on the effectiveness, safety and efficiency of many clinical, public health and social policy choices.”
    Dr. Merrick Zwarenstein, Senior Scientist at ICES Western

  • Between them, Dr. Zwarenstein and Dr. Baiju Shah have led five
    large-scale pragmatic randomized trials using ICES data to evaluate
    the effect of knowledge translation interventions on patients of
    Ontario primary care physicians and home care providers. These studies found that at a large scale, different kinds of printed
    educational materials did not improve family physician adherence to evidence-based care guidelines, and that care pathways
    for home care provision did not improve care outcomes
    for a number of conditions.
  • The ISLAND trial, led by Drs. Noah Ivers and J.D Schwalm, showed that interventions like mail-outs and phone calls encouraged more patients to adhere to their cardiac rehabilitation program after a heart attack.
  • The MyTEMP trial, led by Drs. Ahmed Al-Jaishi and Amit Garg,
    is examining the effect of patients being provided a personalized
    dialysate temperature during each dialysis session on their risk of
    cardiovascular-related death or hospitalization.
  • The CHOICES trial, led by Dr. Jay Udell, uses a multicomponent
    intervention and implementation approach that includes audit and
    feedback reports for family physicians and educational materials
    and tools for patients to help increase the use of cholesterol-lowering
    statin medications.

Posted November 2020

Published in the 2019-20 Annual Report

Download the PDF

Read more impact stories

×