HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Özyürek, A. Articles by Hagoort, P. Search for Related Content PubMed Articles by Özyürek, A. Articles by Hagoort, P.

On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-related Brain Potentials

Asl Özyürek^1,2, Roel M. Willems¹, Sotaro Kita³ and Peter Hagoort^1,2

¹ F. C. Donders Centre for Cognitive Neuroimaging, Nijmegen, ² Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands, ³ University of Birmingham, UK

Reprint requests should be sent to Asl Özyürek, F. C. Donders Centre for Cognitive Neuroimaging, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands, or via e-mail: asliozu{at}mpi.nl.

During language comprehension, listeners use the global semantic representation from previous sentence or discourse context to immediately integrate the meaning of each upcoming word into the unfolding message-level representation. Here we investigate whether communicative gestures that often spontaneously co-occur with speech are processed in a similar fashion and integrated to previous sentence context in the same way as lexical meaning. Event-related potentials were measured while subjects listened to spoken sentences with a critical verb (e.g., knock), which was accompanied by an iconic co-speech gesture (i.e., KNOCK). Verbal and/or gestural semantic content matched or mismatched the content of the preceding part of the sentence. Despite the difference in the modality and in the specificity of meaning conveyed by spoken words and gestures, the latency, amplitude, and topographical distribution of both word and gesture mismatches are found to be similar, indicating that the brain integrates both types of information simultaneously. This provides evidence for the claim that neural processing in language comprehension involves the simultaneous incorporation of information coming from a broader domain of cognition than only verbal semantics. The neural evidence for similar integration of information from speech and gesture emphasizes the tight interconnection between speech and co-speech gestures.

This article has been cited by other articles:

				R. M. Willems, A. Ozyurek, and P. Hagoort Seeing and Hearing Meaning: ERP and fMRI Evidence of Word versus Picture Integration into a Sentence Context J. Cogn. Neurosci., July 1, 2008; 20(7): 1235 - 1249. [Abstract] [Full Text] [PDF]

				J. Blake, D. Myszczyszyn, A. Jokel, and N. Bebiroglu Gestures accompanying speech in specifically language-impaired children and their timing with speech First Language, May 1, 2008; 28(2): 237 - 253. [Abstract] [PDF]