Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies First Results From the RCT DUPLICATE Initiative
Top Cited Papers
- 16 December 2020
- journal article
- research article
- Published by Ovid Technologies (Wolters Kluwer Health) in Circulation
- Vol. 143 (10), 1002-1013
- https://doi.org/10.1161/CIRCULATIONAHA.120.051718
Abstract
BACKGROUND: Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. METHODS: We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication. RESULTS: Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. CONCLUSIONS: Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. REGISTRATION: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.Keywords
This publication has 45 references indexed in Scilit:
- Saxagliptin and Cardiovascular Outcomes in Patients with Type 2 Diabetes MellitusNew England Journal of Medicine, 2013
- Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromesEuropean Heart Journal, 2012
- One‐to‐many propensity score matching in cohort studiesPharmacoepidemiology and Drug Safety, 2012
- Participation in a Clinical Trial Enhances Adherence and Persistence to TreatmentHypertension, 2011
- Beyond the intention-to-treat in comparative effectiveness researchClinical Trials, 2011
- A basic study design for expedited safety signal evaluation based on electronic healthcare dataPharmacoepidemiology and Drug Safety, 2010
- Ticagrelor versus Clopidogrel in Patients with Acute Coronary SyndromesNew England Journal of Medicine, 2009
- Prasugrel versus Clopidogrel in Patients with Acute Coronary SyndromesNew England Journal of Medicine, 2007
- Selective prescribing led to overestimation of the benefits of lipid-lowering drugsJournal of Clinical Epidemiology, 2006
- Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research DesignsNew England Journal of Medicine, 2000