Quantitative methods in syntax/semantics research: A response to Sprouse and Almeida (2013)
- 1 April 2013
- journal article
- Published by Taylor & Francis Ltd in Language and Cognitive Processes
- Vol. 28 (3), 229-240
- https://doi.org/10.1080/01690965.2012.704385
Abstract
Sprouse and Almeida (S&A) present quantitative results that suggest that intuitive judgments utilised in syntax research are generally correct in two-condition comparisons: the sentence type that is presented as “good/grammatical” is usually rated better than the sentence type that is presented as “bad/ungrammatical” in controlled experiments. Although these evaluations of intuitive relative judgments are valuable, they do not justify the use of nonquantitative linguistic methods. We argue that objectivity is a universal value in science that should be adopted by linguistics. In addition, the reliability measures that S&A report are not sufficient for developing sophisticated linguistic theories. Furthermore, quantitative methods yield two additional benefits: consistency of judgments across many pairs of judgments; and an understanding of the relative effect sizes across sets of judgments. We illustrate these points with an experiment that demonstrates five clear levels of acceptability. Finally, we observe that S&A's experiments—where only two authors evaluated 10 years' worth of journal articles and one standard textbook within a few months—further emphasise one of our critical original points: conducting behavioural experiments is in many respects easy and fast with the advent of online research tools like Amazon's Mechanical Turk. Given the current ease of performing quantitative experiments (using a platform like Mechanical Turk) and the clear limitations of not doing so, linguistic hypotheses should be evaluated quantitatively whenever it is feasible.Keywords
This publication has 12 references indexed in Scilit:
- The need for quantitative methods in syntax and semantics researchLanguage and Cognitive Processes, 2013
- A quantitative investigation of the imperative-and-declarative construction in EnglishLanguage, 2011
- Using Mechanical Turk to Obtain and Analyze English Acceptability JudgmentsLanguage and Linguistics Compass, 2011
- Adding a Third Wh-phrase Does Not Increase the Acceptability of Object-initial Multiple-wh-questionsSyntax, 2010
- Weak quantitative standards in linguistics researchTrends in Cognitive Sciences, 2010
- Data in generative grammar: The stick and the carrotTheoretical Linguistics, 2007
- Amnestying Superiority Violations: Processing Multiple QuestionsLinguistic Inquiry, 2006
- Mindless statisticsThe Journal of Socio-Economics, 2004
- The earth is round (p < .05).American Psychologist, 1994
- Dative questions: A study in the relation of acceptability to grammaticality of an english sentence typeCognition, 1973