Bootstrapping in Applied Linguistics: Assessing its Potential Using Shared Data
- 14 February 2014
- journal article
- research article
- Published by Oxford University Press (OUP) in Applied Linguistics
- Vol. 36 (5), 591-610
- https://doi.org/10.1093/applin/amu001
Abstract
Parametric analyses such as t tests and ANOVAs are the norm—if not the default—statistical tests found in quantitative applied linguistics research (Gass 2009). Applied statisticians and one applied linguist (Larson-Hall 2010, 2012; Larson-Hall and Herrington 2010), however, have argued that this approach may not be appropriate for small samples and/or nonnormally distributed data (e.g. Wilcox 2003), both common in second language (L2) research. They recommend instead ‘robust statistics’ such as bootstrapping, a nonparametric procedure that randomly resamples from an observed data set to produce a simulated but more stable and statistically accurate outcome. The present study tests the usefulness of bootstrapping by reanalyzing raw data from 26 studies of applied linguistics research. Our results found no evidence of Type II error (false negative). However, 4 out of 16 statistically significant results were not replicated (i.e. a Type I error ‘misfit’ five times higher than an alpha of .05). We discuss empirically justified suggestions for the use of bootstrapping in the context of broader methodological issues and reforms in applied linguistics (see Plonsky 2013, 2014).This publication has 42 references indexed in Scilit:
- Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical ResultsPLOS ONE, 2011
- Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction ResearchLanguage Learning, 2011
- E2F1 Regulates Cellular Growth by mTORC1 SignalingPLOS ONE, 2011
- Improving Data Analysis in Second Language Acquisition by Utilizing Modern Developments in Applied StatisticsApplied Linguistics, 2009
- Bootstrapping to test for nonzero population correlation coefficients using univariate sampling.Psychological Methods, 2007
- Replication Data Sets and Favored-Hypothesis BiasSociological Methods & Research, 2007
- The poor availability of psychological research data for reanalysis.American Psychologist, 2006
- The Multimedia Adult ESL Learner CorpusTESOL Quarterly, 2003
- Bootstrapping correlation coefficients using univariate and bivariate sampling.Psychological Methods, 1998
- Replication: A View from the StreetsPS: Political Science and Politics, 1995