Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes
Open Access
- 17 June 2012
- journal article
- research article
- Published by Oxford University Press (OUP) in European Heart Journal
- Vol. 33 (15), 1893-1901
- https://doi.org/10.1093/eurheartj/ehs114
Abstract
Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of therapeutic interventions because randomization protects from biases inherent in observational studies. Propensity score (PS) methods, proposed as a potential solution to confounding of the treatment–outcome association, are widely used in observational studies of therapeutic interventions for acute coronary syndromes (ACS). We aimed to systematically assess agreement between observational studies using PS methods and RCTs on therapeutic interventions for ACS. We searched for observational studies of interventions for ACS that used PS methods to estimate treatment effects on short- or long-term mortality. Using a standardized algorithm, we matched observational studies to RCTs based on patients’ characteristics, interventions, and outcomes (‘topics’), and we compared estimates of treatment effect between the two designs. When multiple observational studies or RCTs were identified for the same topic, we performed a meta-analysis and used the summary relative risk for comparisons. We matched 21 observational studies investigating 17 distinct clinical topics to 63 RCTs (median = 3 RCTs per observational study) for short-term (7 topics) and long-term (10 topics) mortality. Estimates from PS analyses differed statistically significantly from randomized evidence in two instances; however, observational studies reported more extreme beneficial treatment effects compared with RCTs in 13 of 17 instances (P = 0.049). Sensitivity analyses limited to large RCTs, and using alternative meta-analysis models yielded similar results. For the treatment of ACS, observational studies using PS methods produce treatment effect estimates that are of more extreme magnitude compared with those from RCTs, although the differences are rarely statistically significant.Keywords
This publication has 31 references indexed in Scilit:
- Heart Disease and Stroke Statistics—2011 UpdateCirculation, 2011
- Multivariate-adjusted pharmacoepidemiologic analyses of confidential information pooled from multiple health care utilization databasesPharmacoepidemiology and Drug Safety, 2010
- Therapeutic Innovations, Diminishing Returns, and Control Rate PreservationJAMA, 2009
- High-dimensional Propensity Score Adjustment in Studies of Treatment Effects Using Health Care Claims DataEpidemiology, 2009
- A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003Statistics in Medicine, 2007
- Analysis of Observational Studies in the Presence of Treatment Selection BiasJAMA, 2007
- A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methodsJournal of Clinical Epidemiology, 2006
- Combining Propensity Score Matching with Additional Adjustments for Prognostic CovariatesJournal of the American Statistical Association, 2000
- Meta-analysis in clinical trialsControlled Clinical Trials, 1986
- The central role of the propensity score in observational studies for causal effectsBiometrika, 1983