On the usefulness of formal judgment tasks in syntax and in second-language research: The case of resumptive pronouns in English, Turkish, and Mandarin Chinese

Abstract

Gibson and Fedorenko (2013, The need for quantitative methods in syntax and semantics research, Language and Cognitive Processes 28(2), 88–124) have argued against the continued use of informally collected acceptability judgments as the primary methodology in theoretical syntax and semantics research. We provide further support for their position with data from Mandarin and Turkish-language judgment tasks which examined the acceptability of resumptive pronouns (RPs) in relative clauses. Based on previous studies which relied on informal judgments, we expected that RPs should be permitted in certain types of Mandarin relative clauses, but ungrammatical in comparable Turkish relative clauses. The results failed to replicate this contrast: RPs were more acceptable than expected in Turkish, and less acceptable than expected in Mandarin. Furthermore, the Mandarin Chinese experiment showed an unexpected gradient effect. We argue that these results challenge existing theoretical accounts, support the more widespread adoption of experimental tasks in theoretical linguistics and in second-language research, and consistently support the Filler-Gap Domain complexity ranking as proposed by Hawkins (2004, Efficiency and complexity in grammars, Oxford: Oxford University Press). We use the complexity ranking and its supporting evidence as a case study demonstrating that quantitative data, such as the evidence obtained from formal sentence judgment tasks, are indispensable in the defense or criticism of linguistic theories.