HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Giard, M. H. Articles by Peronnet, F. Search for Related Content PubMed PubMed Citation Articles by Giard, M. H. Articles by Peronnet, F.

Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study

^a Mental Processes and Brain Activation, INSERM-U280, Lyon, France

The aim of this study was (1) to provide behavioral evidence for multimodal feature integration in an object recognition task in humans and (2) to characterize the processing stages and the neural structures where multisensory interactions take place. Event-related potentials (ERPs) were recorded from 30 scalp electrodes while subjects performed a forced-choice reaction-time categorization task: At each trial, the subjects had to indicate which of two objects was presented by pressing one of two keys. The two objects were defined by auditory features alone, visual features alone, or the combination of auditory and visual features. Subjects were more accurate and rapid at identifying multimodal than unimodal objects. Spatiotemporal analysis of ERPs and scalp current densities revealed several auditory-visual interaction components temporally, spatially, and functionally distinct before 200 msec poststimulus. The effects observed were (1) in visual areas, new neural activities (as early as 40 msec poststimulus) and modulation (amplitude decrease) of the N185 wave to unimodal visual stimulus, (2) in the auditory cortex, modulation (amplitude increase) of subcomponents of the unimodal auditory N1 wave around 90 to 110 msec, and (3) new neural activity over the right fronto-temporal area (140 to 165 msec). Furthermore, when the subjects were separated into two groups according to their dominant modality to perform the task in unimodal conditions (shortest reaction time criteria), the integration effects were found to be similar for the two groups over the nonspecific fronto-temporal areas, but they clearly differed in the sensory-specific cortices, affecting predominantly the sensory areas of the nondominant modality. Taken together, the results indicate that multisensory integration is mediated by flexible, highly adaptive physiological processes that can take place very early in the sensory processing chain and operate in both sensory-specific and nonspecific cortical structures in different ways.

This article has been cited by other articles:

				D. S. Kislyuk, R. Mottonen, and M. Sams Visual Processing Affects the Neural Basis of Auditory Discrimination J. Cogn. Neurosci., December 1, 2008; 20(12): 2175 - 2184. [Abstract] [Full Text] [PDF]

				W. E. HUDDLESTON, J. W. LEWIS, R. E. PHINNEY JR., and E. A. DEYOE Auditory and visual attention-based apparent motion share functional parallels Percept Psychophys, October 1, 2008; 70(7): 1207 - 1216. [Abstract] [PDF]

				J. Hocking and C. J. Price The Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing Cereb Cortex, October 1, 2008; 18(10): 2439 - 2449. [Abstract] [Full Text] [PDF]

				O. Collignon, M. Davare, A. G. De Volder, C. Poirier, E. Olivier, and C. Veraart Time-course of Posterior Parietal and Occipital Cortex Contribution to Sound Localization J. Cogn. Neurosci., August 1, 2008; 20(8): 1454 - 1463. [Abstract] [Full Text] [PDF]

				C. Kayser, C. I. Petkov, and N. K. Logothetis Visual Modulation of Neurons in Auditory Cortex Cereb Cortex, July 1, 2008; 18(7): 1560 - 1574. [Abstract] [Full Text] [PDF]

				G. F. Alpert, G. Hein, N. Tsai, M. J. Naumer, and R. T. Knight Temporal Characteristics of Audiovisual Information Processing J. Neurosci., May 14, 2008; 28(20): 5344 - 5349. [Abstract] [Full Text] [PDF]

				J. J. Stekelenburg and J. Vroomen Neural correlates of multisensory integration of ecologically valid audiovisual events. J. Cogn. Neurosci., December 1, 2007; 19(12): 1964 - 1973. [Abstract] [Full Text] [PDF]

				V. Romei, M. M. Murray, L. B. Merabet, and G. Thut Occipital Transcranial Magnetic Stimulation Has Opposing Effects on Visual and Auditory Stimulus Detection: Implications for Multisensory Interactions J. Neurosci., October 24, 2007; 27(43): 11465 - 11472. [Abstract] [Full Text] [PDF]

				T. Noesselt, J. W. Rieger, M. A. Schoenfeld, M. Kanowski, H. Hinrichs, H.-J. Heinze, and J. Driver Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices J. Neurosci., October 17, 2007; 27(42): 11431 - 11441. [Abstract] [Full Text] [PDF]

				R. F. Reynolds and B. L. Day Fast visuomotor processing made faster by sound J. Physiol., September 15, 2007; 583(3): 1107 - 1115. [Abstract] [Full Text] [PDF]

				J. K. Bizley, F. R. Nodal, V. M. Bajo, I. Nelken, and A. J. King Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex Cereb Cortex, September 1, 2007; 17(9): 2172 - 2189. [Abstract] [Full Text] [PDF]

				R. Martuzzi, M. M. Murray, C. M. Michel, J.-P. Thiran, P. P. Maeder, S. Clarke, and R. A. Meuli Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics Cereb Cortex, July 1, 2007; 17(7): 1672 - 1679. [Abstract] [Full Text] [PDF]

				O. Baumann and M. W. Greenlee Neural Correlates of Coherent Audiovisual Motion Perception Cereb Cortex, June 1, 2007; 17(6): 1433 - 1443. [Abstract] [Full Text] [PDF]

				J. Mishra, A. Martinez, T. J. Sejnowski, and S. A. Hillyard Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion J. Neurosci., April 11, 2007; 27(15): 4120 - 4131. [Abstract] [Full Text] [PDF]

				D. Talsma, T. J. Doty, and M. G. Woldorff Selective Attention and Audiovisual Integration: Is Attending to Both Modalities a Prerequisite for Early Integration? Cereb Cortex, March 1, 2007; 17(3): 679 - 690. [Abstract] [Full Text] [PDF]

				C. Kayser, C. I. Petkov, M. Augath, and N. K. Logothetis Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex J. Neurosci., February 21, 2007; 27(8): 1824 - 1835. [Abstract] [Full Text] [PDF]

				M. Avillac, S. Ben Hamed, and J.-R. Duhamel Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey J. Neurosci., February 21, 2007; 27(8): 1922 - 1932. [Abstract] [Full Text] [PDF]

				K. E. Weaver and A. A. Stevens Attention and Sensory Interactions within the Occipital Cortex in the Early Blind: An fMRI Study. J. Cogn. Neurosci., February 1, 2007; 19(2): 315 - 330. [Abstract] [Full Text] [PDF]

				S. Yuval-Greenberg and L. Y. Deouell What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition J. Neurosci., January 31, 2007; 27(5): 1090 - 1096. [Abstract] [Full Text] [PDF]

				D. Senkowski, S. Molholm, M. Gomez-Ramirez, and J. J. Foxe Oscillatory Beta Activity Predicts Response Speed during a Multisensory Audiovisual Reaction Time Task: A High-Density Electrical Mapping Study Cereb Cortex, November 1, 2006; 16(11): 1556 - 1565. [Abstract] [Full Text] [PDF]

				E. Macaluso Multisensory Processing in Sensory-Specific Cortical Areas Neuroscientist, August 1, 2006; 12(4): 327 - 338. [Abstract] [PDF]

				S. Molholm, P. Sehatpour, A. D. Mehta, M. Shpaner, M. Gomez-Ramirez, S. Ortigue, J. P. Dyke, T. H. Schwartz, and J. J. Foxe Audio-Visual Multisensory Integration in Superior Parietal Lobule Revealed by Human Intracranial Recordings J Neurophysiol, August 1, 2006; 96(2): 721 - 729. [Abstract] [Full Text] [PDF]

				W. A. Teder-Salejarvi, F. Di Russo, J. J. McDonald, and S. A. Hillyard Effects of Spatial Congruity on Audio-Visual Multimodal Integration J. Cogn. Neurosci., September 1, 2005; 17(9): 1396 - 1409. [Abstract] [Full Text] [PDF]

				F. Frassinetti, N. Bolognini, D. Bottari, A. Bonora, and E. Ladavas Audiovisual Integration in Patients with Visual Deficit J. Cogn. Neurosci., September 1, 2005; 17(9): 1442 - 1452. [Abstract] [Full Text] [PDF]

				D. Talsma and M. G. Woldorff Selective Attention and Multisensory Integration: Multiple Phases of Effects on the Evoked Brain Activity J. Cogn. Neurosci., July 1, 2005; 17(7): 1098 - 1114. [Abstract] [Full Text] [PDF]

				M. M. Murray, S. Molholm, C. M. Michel, D. J. Heslenfeld, W. Ritter, D. C. Javitt, C. E. Schroeder, and J. J. Foxe Grabbing Your Ear: Rapid Auditory-Somatosensory Multisensory Interactions in Low-level Sensory Cortices Are Not Constrained by Stimulus Alignment Cereb Cortex, July 1, 2005; 15(7): 963 - 974. [Abstract] [Full Text] [PDF]

				L. M. Miller and M. D'Esposito Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech J. Neurosci., June 22, 2005; 25(25): 5884 - 5893. [Abstract] [Full Text] [PDF]

				Y. Feng and T. L. Simpson The Inhibitory Interaction between Human Corneal and Conjunctival Sensory Channels Invest. Ophthalmol. Vis. Sci., April 1, 2005; 46(4): 1251 - 1255. [Abstract] [Full Text] [PDF]

				N. E. Barraclough, D. Xiao, C. I. Baker, M. W. Oram, and D. I. Perrett Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions J. Cogn. Neurosci., March 1, 2005; 17(3): 377 - 391. [Abstract] [Full Text] [PDF]

				V. van Wassenhove, K. W. Grant, and D. Poeppel Visual speech speeds up the neural processing of auditory speech PNAS, January 25, 2005; 102(4): 1181 - 1186. [Abstract] [Full Text] [PDF]

				D. E. Callan, J. A. Jones, K. Munhall, C. Kroos, A. M. Callan, and E. Vatikiotis-Bateson Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information J. Cogn. Neurosci., June 1, 2004; 16(5): 805 - 816. [Abstract] [Full Text] [PDF]

				S. Molholm, W. Ritter, D. C. Javitt, and J. J. Foxe Multisensory Visual-Auditory Object Recognition in Humans: a High-density Electrical Mapping Study Cereb Cortex, April 1, 2004; 14(4): 452 - 465. [Abstract] [Full Text] [PDF]

				M. T. Wallace, R. Ramachandran, and B. E. Stein A revised view of sensory cortical parcellation PNAS, February 17, 2004; 101(7): 2167 - 2172. [Abstract] [Full Text] [PDF]

				W. Jiang and B. E. Stein Cortex Controls Multisensory Depression in Superior Colliculus J Neurophysiol, October 1, 2003; 90(4): 2123 - 2135. [Abstract] [Full Text] [PDF]

				K.-M. G. Fu, T. A. Johnston, A. S. Shah, L. Arnold, J. Smiley, T. A. Hackett, P. E. Garraghty, and C. E. Schroeder Auditory Cortical Neurons Respond to Somatosensory Stimulation J. Neurosci., August 20, 2003; 23(20): 7510 - 7515. [Abstract] [Full Text] [PDF]

				W. D. Hairston, M. T. Wallace, J.W. Vaughan, B. E. Stein, J. L. Norris, and J. A. Schirillo Visual Localization Ability Influences Cross-Modal Bias J. Cogn. Neurosci., January 1, 2003; 15(1): 20 - 29. [Abstract] [Full Text] [PDF]

				W. Jiang, H. Jiang, and B. E. Stein Two Corticotectal Areas Facilitate Multisensory Orientation Behavior J. Cogn. Neurosci., November 1, 2002; 14(8): 1240 - 1255. [Abstract] [Full Text] [PDF]

				A. Fort, C. Delpuech, J. Pernier, and M.-H. Giard Dynamics of Cortico-subcortical Cross-modal Operations Involved in Audio-visual Object Detection in Humans Cereb Cortex, October 1, 2002; 12(10): 1031 - 1039. [Abstract] [Full Text] [PDF]

				J. J. Foxe, G. R. Wylie, A. Martinez, C. E. Schroeder, D. C. Javitt, D. Guilfoyle, W. Ritter, and M. M. Murray Auditory-Somatosensory Multisensory Processing in Auditory Association Cortex: An fMRI Study J Neurophysiol, July 1, 2002; 88(1): 540 - 543. [Abstract] [Full Text] [PDF]

				A. Falchier, S. Clavagnier, P. Barone, and H. Kennedy Anatomical Evidence of Multimodal Integration in Primate Striate Cortex J. Neurosci., July 1, 2002; 22(13): 5749 - 5759. [Abstract] [Full Text] [PDF]

				P. J. Laurienti, J. H. Burdette, M. T. Wallace, Y.-F. Yen, A. S. Field, and B. E. Stein Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli J. Cogn. Neurosci., April 1, 2002; 14(3): 420 - 429. [Abstract] [Full Text] [PDF]

				B. de Gelder, G. Pourtois, and L. Weiskrantz Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures PNAS, March 19, 2002; 99(6): 4121 - 4126. [Abstract] [Full Text] [PDF]

				G. A. Calvert Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies Cereb Cortex, December 1, 2001; 11(12): 1110 - 1123. [Abstract] [Full Text] [PDF]

				M. T. Wallace and B. E. Stein Sensory and Multisensory Responses in the Newborn Monkey Superior Colliculus J. Neurosci., November 15, 2001; 21(22): 8886 - 8894. [Abstract] [Full Text] [PDF]