HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Callan, D. E. Articles by Vatikiotis-Bateson, E. Search for Related Content PubMed PubMed Citation Articles by Callan, D. E. Articles by Vatikiotis-Bateson, E.

Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information

ATR International

Jeffery A. Jones

ATR International
Wilfrid Laurier University

Kevin Munhall

ATR International
Queens University

Christian Kroos

Munich University

Akiko M. Callan

ATR International

Eric Vatikiotis-Bateson

ATR International
University of British Columbia

Perception of speech is improved when presentation of the audio signal is accompanied by concordant visual speech gesture information. This enhancement is most prevalent when the audio signal is degraded. One potential means by which the brain affords perceptual enhancement is thought to be through the integration of concordant information from multiple sensory channels in a common site of convergence, multisensory integration (MSI) sites. Some studies have identified potential sites in the superior temporal gyrus/sulcus (STG/S) that are responsive to multisensory information from the auditory speech signal and visual speech movement. One limitation of these studies is that they do not control for activity resulting from attentional modulation cued by such things as visual information signaling the onsets and offsets of the acoustic speech signal, as well as activity resulting from MSI of properties of the auditory speech signal with aspects of gross visual motion that are not specific to place of articulation information. This fMRI experiment uses spatial wavelet band-pass filtered Japanese sentences presented with background multispeaker audio noise to discern brain activity reflecting MSI induced by auditory and visual correspondence of place of articulation information that controls for activity resulting from the above-mentioned factors. The experiment consists of a low-frequency (LF) filtered condition containing gross visual motion of the lips, jaw, and head without specific place of articulation information, a midfrequency (MF) filtered condition containing place of articulation information, and an unfiltered (UF) condition. Sites of MSI selectively induced by auditory and visual correspondence of place of articulation information were determined by the presence of activity for both the MF and UF conditions relative to the LF condition. Based on these criteria, sites of MSI were found predominantly in the left middle temporal gyrus (MTG), and the left STG/S (including the auditory cortex). By controlling for additional factors that could also induce greater activity resulting from visual motion information, this study identifies potential MSI sites that we believe are involved with improved speech perception intelligibility.

This article has been cited by other articles:

				U. Noppeney, O. Josephs, J. Hocking, C. J. Price, and K. J. Friston The Effect of Prior Visual Information on Recognition of Speech and Sounds Cereb Cortex, March 1, 2008; 18(3): 598 - 609. [Abstract] [Full Text] [PDF]

				J. J. Stekelenburg and J. Vroomen Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci., December 1, 2007; 19(12): 1964 - 1973. [Abstract] [Full Text] [PDF]

				L. M. Miller and M. D'Esposito Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech J. Neurosci., June 22, 2005; 25(25): 5884 - 5893. [Abstract] [Full Text] [PDF]