Language as a multimodal phenomenon: implications for language learning, processing and evolution
- 19 September 2014
- journal article
- editorial
- Published by The Royal Society in Philosophical Transactions B
- Vol. 369 (1651), 20130292
- https://doi.org/10.1098/rstb.2013.0292
Abstract
Our understanding of the cognitive and neural underpinnings of language has traditionally been firmly based on spoken Indo-European languages and on language studied as speech or text. However, in face-to-face communication, language is multimodal: speech signals are invariably accompanied by visual information on the face and in manual gestures, and sign languages deploy multiple channels (hands, face and body) in utterance construction. Moreover, the narrow focus on spoken Indo-European languages has entrenched the assumption that language is comprised wholly by an arbitrary system of symbols and rules. However, iconicity (i.e. resemblance between aspects of communicative form and meaning) is also present: speakers use iconic gestures when they speak; many non-Indo-European spoken languages exhibit a substantial amount of iconicity in word forms and, finally, iconicity is the norm, rather than the exception in sign languages. This introduction provides the motivation for taking a multimodal approach to the study of language learning, processing and evolution, and discusses the broad implications of shifting our current dominant approaches and assumptions to encompass multimodal expression in both signed and spoken languages.Keywords
This publication has 51 references indexed in Scilit:
- Investigation of the process underpinning vowel‐size correspondenceJapanese Psychological Research, 2013
- Iconicity as a General Property of Language: Evidence from Spoken and Signed LanguagesFrontiers in Psychology, 2010
- Effects of iconicity and semantic relatedness on lexical access in american sign language.Journal of Experimental Psychology: Learning, Memory, and Cognition, 2010
- Gesturing Gives Children New Ideas About MathPsychological Science, 2009
- The link between form and meaning in American Sign Language: Lexical processing effects.Journal of Experimental Psychology: Learning, Memory, and Cognition, 2009
- Differences in Early Gesture Explain SES Disparities in Child Vocabulary Size at School EntryScience, 2009
- Symbiotic symbolization by hand and mouth in sign languageSemiotica, 2009
- The shape of boubas: sound–shape correspondences in toddlers and adultsDevelopmental Science, 2006
- Implicit sound symbolism in lexical access: Evidence from an interference taskBrain and Language, 2005
- Hearing lips and seeing voicesNature, 1976