Outcomes and Statistical Power in Adult Critical Care Randomized Trials
- 15 June 2014
- journal article
- Published by American Thoracic Society in American Journal of Respiratory and Critical Care Medicine
- Vol. 189 (12), 1469-1478
- https://doi.org/10.1164/rccm.201401-0056cp
Abstract
Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits. Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.Keywords
This publication has 51 references indexed in Scilit:
- Hydroxyethyl Starch or Saline for Fluid Resuscitation in Intensive CareThe New England Journal of Medicine, 2012
- Hydroxyethyl Starch 130/0.42 versus Ringer's Acetate in Severe SepsisThe New England Journal of Medicine, 2012
- A stepwise approach to justify phase III randomized clinical trials and enhance the likelihood of a positive resultCritical Care Medicine, 2010
- Beyond MortalityAmerican Journal of Respiratory and Critical Care Medicine, 2010
- A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT ProjectClinical Trials, 2010
- Underpowered trials in critical care medicine: how to deal with them?Critical Care, 2010
- Delta inflation: a bias in the design of randomized controlled trials in critical care medicineCritical Care, 2010
- Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics supportJournal of Biomedical Informatics, 2008
- Approaches for estimating prevalence ratiosOccupational and Environmental Medicine, 2008
- Competing risk and heterogeneity of treatment effect in clinical trialsTrials, 2008