Online citations, reference lists, and bibliographies.
Referencing for people who value simplicity, privacy, and speed.
Get Citationsy
← Back to Search

Electrocorticography Reveals Enhanced Visual Cortex Responses To Visual Speech.

I. Schepers, D. Yoshor, M. Beauchamp
Published 2015 · Medicine, Psychology

Save to my Library
Download PDF
Analyze on Scholarcy Visualize in Litmaps
Share
Reduce the time it takes to create your bibliography by a factor of 10 by using the world’s favourite reference manager
Time to take this seriously.
Get Citationsy
Human speech contains both auditory and visual components, processed by their respective sensory cortices. We test a simple model in which task-relevant speech information is enhanced during cortical processing. Visual speech is most important when the auditory component is uninformative. Therefore, the model predicts that visual cortex responses should be enhanced to visual-only (V) speech compared with audiovisual (AV) speech. We recorded neuronal activity as patients perceived auditory-only (A), V, and AV speech. Visual cortex showed strong increases in high-gamma band power and strong decreases in alpha-band power to V and AV speech. Consistent with the model prediction, gamma-band increases and alpha-band decreases were stronger for V speech. The model predicts that the uninformative nature of the auditory component (not simply its absence) is the critical factor, a prediction we tested in a second experiment in which visual speech was paired with auditory white noise. As predicted, visual speech with auditory noise showed enhanced visual cortex responses relative to AV speech. An examination of the anatomical locus of the effects showed that all visual areas, including primary visual cortex, showed enhanced responses. Visual cortex responses to speech are enhanced under circumstances when visual information is most important for comprehension.
This paper references
10.1093/CERCOR/9.5.415
Electrophysiological studies of human face perception. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli.
T. Allison (1999)
10.1016/j.schres.2007.08.008
Impaired multisensory processing in schizophrenia: Deficits in the visual enhancement of speech comprehension under noisy environmental conditions
Lars A. Ross (2007)
10.3389/neuro.07.007.2009
Directed Interactions Between Auditory and Superior Temporal Cortices and their Role in Sensory Integration
C. Kayser (2009)
10.1523/JNEUROSCI.4853-10.2011
Dynamic Changes in Superior Temporal Sulcus Connectivity during Perception of Noisy Audiovisual Speech
A. Nath (2011)
10.1016/j.tics.2005.03.009
Speech-specific auditory processing: where is it?
C. Price (2005)
10.1111/j.1469-8986.2011.01273.x
Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review.
David M. Groppe (2011)
10.1113/jphysiol.1968.sp008455
Receptive fields and functional architecture of monkey striate cortex
D. Hubel (1968)
Cortical surface-based analysis
AM Dale (1999)
10.1016/S1053-8119(09)71265-0
Speech Experience Shapes the Speechreading Network and Subsequent Deafness Facilitates It.
H. Lee (2009)
10.1111/j.1460-9568.2004.03670.x
Bimodal speech: early suppressive visual effects in human auditory cortex
Julien Besle (2004)
10.1016/j.neuroimage.2013.04.017
fMRI hemodynamics accurately reflects neuronal timing in the human brain measured by MEG
Fa-Hsuan Lin (2013)
10.1097/00003446-200504000-00004
Development of Audiovisual Comprehension Skills in Prelingually Deaf Children With Cochlear Implants
T. Bergeson (2005)
10.1523/JNEUROSCI.6510-10.2011
Physical and Perceptual Factors Shape the Neural Mechanisms That Integrate Audiovisual Signals in Speech Comprehension
H. Lee (2011)
10.1093/brain/aws014
Functionally specific oscillatory activity correlates between visual and auditory cortex in the blind.
I. Schepers (2012)
10.1371/journal.pone.0068959
An fMRI Study of Audiovisual Speech Perception Reveals Multisensory Interactions in Auditory Cortex
K. Okada (2013)
10.1016/j.clinph.2004.11.016
Electrophysiological studies on human pain perception
R. Kakigi (2005)
10.1155/2011/156869
FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data
R. Oostenveld (2011)
ferential selectivity for dynamic versus static information in faceselective cortical regions
C Price (2011)
10.1126/science.1223082
The Pulvinar Regulates Information Transmission Between Cortical Areas Based on Attention Demands
Y. Saalmann (2012)
10.1038/ncomms1715
Ultra-rapid access to words in the brain
L. MacGregor (2012)
10.1038/nn.2641
Categorical Speech Representation in Human Superior Temporal Gyrus
E. Chang (2010)
10.1016/j.neuron.2007.06.004
Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex
H. Luo (2007)
Subcortical, modality-specific pathways
HA Slagter (2013)
10.1093/CERCOR/BHK025
High-frequency activity in human visual cortex is modulated by visual motion strength.
M. Siegel (2007)
10.1371/journal.pone.0046339
Temporal Sequence of Visuo-Auditory Interaction in Multiple Areas of the Guinea Pig Visual Cortex
M. Nishimura (2012)
10.1038/nn909
Stages of processing in face perception: an MEG study
J. Liu (2002)
10.1371/journal.pone.0019550
The Role of Gamma-Band Activity in the Representation of Faces: Reduced Activity in the Fusiform Face Area in Congenital Prosopagnosia
C. Dobel (2011)
10.1038/nrn2811
Active perception: sensorimotor circuits as a cortical basis for language
F. Pulvermüller (2010)
10.1214/AOS/1013699998
THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY
Y. Benjamini (2001)
10.1093/cercor/bht069
Subcortical, modality-specific pathways contribute to multisensory processing in humans.
R. L. van den Brink (2014)
10.1016/j.neuroimage.2013.04.060
Auditory-driven phase reset in visual cortex: Human electrocorticography reveals mechanisms of early multisensory integration
M. Mercier (2013)
10.1523/JNEUROSCI.22-13-05749.2002
Anatomical Evidence of Multimodal Integration in Primate Striate Cortex
A. Falchier (2002)
10.1121/1.1907309
Visual contribution to speech intelligibility in noise
W. H. Sumby (1954)
10.1093/CERCOR/BHM049
Beyond superior temporal cortex: intersubject correlations in narrative speech comprehension.
S. Wilson (2008)
10.1093/CERCOR/BHH166
Hearing lips: gamma-band activity during audiovisual speech perception.
J. Kaiser (2005)
10.1038/nrn2113
The cortical organization of speech processing
G. Hickok (2007)
10.1371/journal.pbio.1000610
Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex
S. Ray (2011)
10.1016/j.conb.2012.11.011
Cortical circuits for the control of attention
E. Miller (2013)
FieldTrip: open
R Oostenveld (2011)
10.1016/j.neuropsychologia.2012.09.031
Spatial attention modulates early face processing
Wenfeng Feng (2012)
Cortical surface-based analysis
B Fischl (1999)
10.1016/j.neuroimage.2013.04.064
Audio–visual congruency alters power and coherence of oscillatory activity within and between cortical areas
J. Lange (2013)
10.1007/978-1-4615-6616-8
Cerebral Cortex
E. Jones (1987)
10.1093/CERCOR/10.1.69
Cognitive response profile of the human fusiform face area as determined by MEG.
E. Halgren (2000)
10.1006/nimg.1998.0395
Cortical Surface-Based Analysis: I. Segmentation and Surface Reconstruction
A. Dale (1999)
10.1016/S0896-6273(04)00070-4
Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus
M. Beauchamp (2004)
10.1093/cercor/bhp082
Connections of auditory and visual cortex in the prairie vole (Microtus ochrogaster): evidence for multisensory processing in primary sensory areas.
K. Campi (2010)
10.1016/j.conb.2007.07.003
The thalamus is more than just a relay
S. Sherman (2007)
10.1146/annurev-neuro-060909-152823
Attention, intention, and priority in the parietal lobe.
J. Bisley (2010)
10.1121/1.4781387
Audiovisual integration and lipreading abilities of older adults with normal and impaired hearing.
N. Tye‐Murray (2007)
10.1162/089892999563544
Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study
M. H. Giard (1999)
10.1002/1096-9861(20001204)428:1<112::AID-CNE8>3.0.CO;2-9
Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey
J. Lewis (2000)
10.1016/S0926-6410(02)00066-6
Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study.
S. Molholm (2002)
10.3389/CONF.FNHUM.2011.207.00307
Oscillatory synchronization in large-scale cortical networks predicts perception
S. Markus (2011)
10.3389/fnhum.2012.00305
The effects of visual imagery on face identification: an ERP study
J. Wu (2012)
Speech experience
MW Suh (2009)
The effects of visual imagery
H Duan (2012)
10.1016/j.neuron.2011.12.026
Sound-Driven Synaptic Inhibition in Primary Visual Cortex
Giuliano Iurilli (2012)
10.1016/j.neulet.2005.03.050
Recruitment of fusiform face area associated with listening to degraded speech sounds in auditory–visual speech perception: a PET study
T. Kawase (2005)
ferential selectivity for dynamic versus static information in faceselective cortical regions
D Pitcher (2011)
10.1016/j.neuroimage.2012.11.066
Noise alters beta-band activity in superior temporal cortex during audiovisual speech processing
I. Schepers (2013)
10.1523/JNEUROSCI.2939-07.2007
Top–Down Attentional Deficits in Macaques with Lesions of Lateral Prefrontal Cortex
Andrew F. Rossi (2007)
10.1016/S1053-8119(09)70632-9
Perceiving Electrical Stimulation of Identified Human Visual Areas
DK Murphey (2009)
Speech - speci fi c auditory processing : where is it ?
F Pulvermuller (2005)
10.1093/CERCOR/BHL024
Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments.
Lars A. Ross (2007)
10.1523/JNEUROSCI.0896-05.2005
Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
L. M. Miller (2005)
Screening and Management of Adult Hearing Loss in Primary Care
S. Bogardus (2003)
10.1016/j.specom.2004.10.011
Auditory speech detection in noise enhanced by lipreading
L. Bernstein (2004)
10.1093/cercor/bhs106
What the Human Brain Likes About Facial Motion
J. Schultz (2013)
289:1976–1985. 8 Electrocorticography Reveals Enhanced Visual Cortex Responses
review. JAMA (1985)
10.1080/14626269409408338
What is a Human?
R. Pepperell (1994)
10.3389/fnhum.2013.00088
Deficits in high- (>60 Hz) gamma-band oscillations during visual processing in schizophrenia
C. Grützner (2013)
Temporal sequence of visuo-auditory
WJ Song (2012)
10.1146/annurev-neuro-062111-150439
Prefrontal contributions to visual selective attention.
Ryan Squire (2013)
10.1001/JAMA.289.15.1976
Screening and management of adult hearing loss in primary care: scientific review.
B. Yueh (2003)
10.1523/JNEUROSCI.3181-12.2013
Spatial and Object-Based Attention Modulates Broadband High-Frequency Responses across the Human Visual Cortical Hierarchy
I. Davidesco (2013)
Speech - speci fi c auditory processing : where is it ?
C Price (2005)
10.1523/JNEUROSCI.0962-09.2009
How Ongoing Fluctuations in Human Visual Cortex Predict Perceptual Awareness: Baseline Shift versus Decision Bias
V. Wyart (2009)
10.1016/j.pneurobio.2012.06.008
High-frequency neural activity and human cognition: Past, present and possible future of intracranial EEG research
J. Lachaux (2012)
10.1016/j.neuroimage.2011.03.067
Differential selectivity for dynamic versus static information in face-selective cortical regions
D. Pitcher (2011)
10.1097/00003446-199008000-00001
Hearing in the elderly: the Framingham cohort, 1983-1985. Part I. Basic audiometric test results.
G. Gates (1990)
10.1016/j.neuroimage.2005.08.043
Localizing human visual gamma-band activity in frequency, time and space
N. Hoogenboom (2006)
10.1006/nimg.1998.0396
Cortical Surface-Based Analysis: II: Inflation, Flattening, and a Surface-Based Coordinate System
B. Fischl (1999)
Dynamic changes in superior
AR Nath (2011)
10.1073/PNAS.0510124103
Dissociation of face-selective cortical responses by attention.
M. Furey (2006)
10.1016/j.neuroimage.2013.03.040
Whole-head rapid fMRI acquisition using echo-shifted magnetic resonance inverse imaging
Wei-Tang Chang (2013)
The pulvinar
MA Pinsk (2012)
10.1002/hbm.20158
Simplified intersubject averaging on the cortical surface using SUMA
B. D. Argall (2006)
10.1093/cercor/bhs078
Effects of parietal TMS on visual and auditory processing at the primary cortical level -- a concurrent TMS-fMRI study.
Joana Leitão (2013)



This paper is referenced by
10.7554/eLife.30387
Frontal cortex selects representations of the talker’s mouth to aid in speech perception
Muge Ozker (2018)
10.1523/JNEUROSCI.0555-20.2020
Crossmodal Phase Reset and Evoked Responses Provide Complementary Mechanisms for the Influence of Visual Speech in Auditory Cortex
P. Mégevand (2020)
10.7554/eLife.24763
Contributions of local speech encoding and functional connectivity to audio-visual speech perception
Bruno L. Giordano (2017)
10.1007/s00429-019-01978-7
Grey matter volume in developmental speech and language disorder
Lauren Pigdon (2019)
10.1111/ejn.13992
Electrocorticography reveals continuous auditory and visual speech tracking in temporal and occipital cortex
Cristiano Micheli (2018)
10.1016/j.neuropsychologia.2015.08.011
The multisensory function of the human primary visual cortex
M. Murray (2016)
The multisensory function of primary visual cortex in humans
M. Murray (2015)
10.1101/626259
Cross-modal Suppression of Auditory Association Cortex by Visual Speech as a Mechanism for Audiovisual Speech Perception
Patrick J. Karas (2019)
Visual speech enhancement and its application in speech perception training
Najwa Alghamdi (2017)
10.1101/2020.04.16.045716
Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution
Brian A. Metzger (2020)
10.7554/eLife.48116
The visual speech head start improves perception and reduces superior temporal cortex responses to auditory speech
Patrick J Karas (2019)
10.1162/jocn_a_01110
A Double Dissociation between Anterior and Posterior Superior Temporal Gyrus for Processing Audiovisual Speech Demonstrated by Electrocorticography
Muge Ozker (2017)
10.1007/s00429-017-1477-0
Neuroanatomical and resting state EEG power correlates of central hearing loss in older adults
Nathalie Giroud (2017)
Learning Nonlinear Dynamical Networks in Neural Systems
Mojtaba Sahraee Ardakan (2016)
10.1523/JNEUROSCI.0279-20.2020
Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution
Brian A. Metzger (2020)
10.1101/097493
Contributions of local speech encoding and functional connectivity to audio-visual speech integration
Bruno L. Giordano (2016)
10.1111/ejn.14425
Look at me when I'm talking to you: Selective attention at a multisensory cocktail party can be decoded using stimulus reconstruction and alpha power modulations
Aisling E. O’Sullivan (2019)
10.1101/405597
Phase resetting in human auditory cortex to visual speech
P. Mégevand (2018)
10.5167/uzh-139234
Dynamics of electrophysiology and morphology in older adults with age-related hearing loss
Nathalie Giroud (2017)
10.3389/fnhum.2016.00679
Visual Cortical Entrainment to Motion and Categorical Speech Features during Silent Lipreading
Aisling E. O’Sullivan (2017)
Investigating the Neural Basis of Audiovisual Speech Perception with Intracranial Recordings in Humans
Muge O Sertel (2017)
10.1080/23273798.2015.1101145
Can you hear me yet? An intracranial investigation of speech and non-speech audiovisual interactions in human cortex
Ariane E. Rhone (2016)
10.1109/BigData50022.2020.9378462
Interpretable Visualization and Higher-Order Dimension Reduction for ECoG Data
Kelly Geyer (2020)
Semantic Scholar Logo Some data provided by SemanticScholar