Challenges to validity in single‐group interrupted time series analysis
- 14 September 2016
- journal article
- Published by Wiley in Journal of Evaluation in Clinical Practice
- Vol. 23 (2), 413-418
- https://doi.org/10.1111/jep.12638
Abstract
Single-group interrupted time series analysis (ITSA) is a popular evaluation methodology in which a single unit of observation is studied; the outcome variable is serially ordered as a time series, and the intervention is expected to “interrupt” the level and/or trend of the time series, subsequent to its introduction. The most common threat to validity is history—the possibility that some other event caused the observed effect in the time series. Although history limits the ability to draw causal inferences from single ITSA models, it can be controlled for by using a comparable control group to serve as the counterfactual. Time series data from 2 natural experiments (effect of Florida's 2000 repeal of its motorcycle helmet law on motorcycle fatalities and California's 1988 Proposition 99 to reduce cigarette sales) are used to illustrate how history biases results of single-group ITSA results—as opposed to when that group's results are contrasted to those of a comparable control group. In the first example, an external event occurring at the same time as the helmet repeal appeared to be the cause of a rise in motorcycle deaths, but was only revealed when Florida was contrasted with comparable control states. Conversely, in the second example, a decreasing trend in cigarette sales prior to the intervention raised question about a treatment effect attributed to Proposition 99, but was reinforced when California was contrasted with comparable control states. Results of single-group ITSA should be considered preliminary, and interpreted with caution, until a more robust study design can be implemented.Keywords
This publication has 27 references indexed in Scilit:
- Assessing regression to the mean effects in health care initiativesBMC Medical Research Methodology, 2013
- Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterpriseClinical and Translational Medicine, 2013
- Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control ProgramJournal of the American Statistical Association, 2010
- Estimating the Effect of Regression to the Mean in Health Management ProgramsDisease Management and Health Outcomes, 2007
- Causal Inference Using Potential OutcomesJournal of the American Statistical Association, 2005
- Evaluating Disease Management Program Effectiveness: An Introduction to Survival AnalysisDisease Management, 2004
- The Effectiveness of State-Level Tobacco Control Interventions: A Review of Program Implementation and Behavioral OutcomesAnnual Review of Public Health, 2002
- A double bootstrap method to analyze linear models with autoregressive error terms.Psychological Methods, 2000
- Analysis of interrupted time series mortality trends: an example to evaluate regionalized perinatal care.American Journal of Public Health, 1981
- Intervention Analysis with Applications to Economic and Environmental ProblemsJournal of the American Statistical Association, 1975