Functional and temporal relations between spoken and gestured components of language
- 21 July 2017
- journal article
- Published by John Benjamins Publishing Company in Corpus Studies of Language Through Time
- Vol. 22 (1), 1-26
- https://doi.org/10.1075/ijcl.22.1.01kok
Abstract
Based on the Bielefeld Speech and Gesture Alignment Corpus (Lücking et al. 2013), this paper presents a systematic comparison of the linguistic characteristics of unimodal (speech only) and multimodal (gesture-accompanied) forms of language use. The results suggest that each of these two modes of expression is characterized by statistical preferences for certain types of words and grammatical categories. The words that are most frequently accompanied by a manual gesture, when controlled for their total frequency, include unspecific spatial lexemes, various deictic words, and particles that express difficulty in word retrieval or formulation. Other linguistic items, including pronouns and verbs of cognition, show a strong dispreference for being gesture-accompanied. The second part of the paper shows that gestures do not occur within a fixed time window relative to the word(s) they relate to, but the preferred temporal distance varies with the type of functional relation that exists between the verbal and gestural channel.Keywords
This publication has 22 references indexed in Scilit:
- The temporal relation between beat gestures and speechLanguage and Cognitive Processes, 2011
- HeadTalk, HandTalk and the corpus: towards a framework for multi-modal, multi-media corpus developmentCorpora, 2009
- Grammars, Words, and Embodied Meanings: On the Uses and Evolution of So and LikeJournal of Communication, 2002
- Lexical gestures and lexical access: a process modelPublished by Cambridge University Press (CUP) ,2000
- Catchments and contexts: non-modular factors in speech and gesture productionPublished by Cambridge University Press (CUP) ,2000
- Iconic gestures: the grammatical categories of lexical affiliatesJournal of Neurolinguistics, 1999
- Gestures as illocutionary and discourse structure markers in Southern Italian conversationJournal of Pragmatics, 1995
- Gesture as communication I: Its coordination with gaze and speechCommunication Monographs, 1993
- Generating and evaluating domain-oriented multi-word terms from textsInformation Processing & Management, 1993
- Speech, gesture, and discourseDiscourse Processes, 1992