Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review
Open Access
- 21 March 2013
- Vol. 346 (mar21 1), f1135
- https://doi.org/10.1136/bmj.f1135
Abstract
Objectives To assess the completeness of reporting of sample size determinations in unpublished research protocols and to develop guidance for research ethics committees and for statisticians advising these committees. Design Review of original research protocols. Study selection Unpublished research protocols for phase IIb, III, and IV randomised clinical trials of investigational medicinal products submitted to research ethics committees in the United Kingdom during 1 January to 31 December 2009. Main outcome measures Completeness of reporting of the sample size determination, including the justification of design assumptions, and disagreement between reported and recalculated sample size. Results 446 study protocols were reviewed. Of these, 190 (43%) justified the treatment effect and 213 (48%) justified the population variability or survival experience. Only 55 (12%) discussed the clinical importance of the treatment effect sought. Few protocols provided a reasoned explanation as to why the design assumptions were plausible for the planned study. Sensitivity analyses investigating how the sample size changed under different design assumptions were lacking; six (1%) protocols included a re-estimation of the sample size in the study design. Overall, 188 (42%) protocols reported all of the information to accurately recalculate the sample size; the assumed withdrawal or dropout rate was not given in 177 (40%) studies. Only 134 of the 446 (30%) sample size calculations could be accurately reproduced. Study size tended to be over-estimated rather than under-estimated. Studies with non-commercial sponsors justified the design assumptions used in the calculation more often than studies with commercial sponsors but less often reported all the components needed to reproduce the sample size calculation. Sample sizes for studies with non-commercial sponsors were less often reproduced. Conclusions Most research protocols did not contain sufficient information to allow the sample size to be reproduced or the plausibility of the design assumptions to be assessed. Greater transparency in the reporting of the determination of the sample size and more focus on study design during the ethical review process would allow deficiencies to be resolved early, before the trial begins. Guidance for research ethics committees and statisticians advising these committees is needed.This publication has 27 references indexed in Scilit:
- Optimism bias leads to inconclusive results—an empirical studyJournal of Clinical Epidemiology, 2011
- Current sample size conventions: Flaws, harms, and alternativesBMC Medicine, 2010
- Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocolsBMJ, 2008
- Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological studyBMJ, 2008
- Waiving confidentiality for the greater goodBMJ, 2006
- Sample sizes for clinical trials with Normal dataStatistics in Medicine, 2004
- A Base-flipping Mechanism for the T4 Phage β-Glucosyltransferase and Identification of a Transition-state AnalogJournal of Molecular Biology, 2002
- Statistical Review by Research Ethics CommitteesJournal of the Royal Statistical Society Series A: Statistics in Society, 2000
- ESTIMATING SAMPLE SIZES FOR CONTINUOUS, BINARY, AND ORDINAL OUTCOMES IN PAIRED COMPARISONS: PRACTICAL HINTSJournal of Biopharmaceutical Statistics, 1999
- Statistics and ethics in medical research: III How large a sample?BMJ, 1980