Visual speech differentially modulates beta, theta, and high gamma bands in auditory cortex

G. Karthik, John Plass, Adriene M. Beltz, Zhongming Liu, Marcia Grabowecky, Satoru Suzuki, William C. Stacey, Vibhangini S. Wasade, Vernon L. Towle, James X. Tao, Shasha Wu, Naoum P. Issa, David Brang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Speech perception is a central component of social communication. Although principally an auditory process, accurate speech perception in everyday settings is supported by meaningful information extracted from visual cues. Visual speech modulates activity in cortical areas subserving auditory speech perception including the superior temporal gyrus (STG). However, it is unknown whether visual modulation of auditory processing is a unitary phenomenon or, rather, consists of multiple functionally distinct processes. To explore this question, we examined neural responses to audiovisual speech measured from intracranially implanted electrodes in 21 patients with epilepsy. We found that visual speech modulated auditory processes in the STG in multiple ways, eliciting temporally and spatially distinct patterns of activity that differed across frequency bands. In the theta band, visual speech suppressed the auditory response from before auditory speech onset to after auditory speech onset (−93 to 500 ms) most strongly in the posterior STG. In the beta band, suppression was seen in the anterior STG from −311 to −195 ms before auditory speech onset and in the middle STG from −195 to 235 ms after speech onset. In high gamma, visual speech enhanced the auditory response from −45 to 24 ms only in the posterior STG. We interpret the visual-induced changes prior to speech onset as reflecting crossmodal prediction of speech signals. In contrast, modulations after sound onset may reflect a decrease in sustained feedforward auditory activity. These results are consistent with models that posit multiple distinct mechanisms supporting audiovisual speech perception.

Original languageEnglish (US)
Pages (from-to)7301-7317
Number of pages17
JournalEuropean Journal of Neuroscience
Volume54
Issue number9
DOIs
StatePublished - Nov 2021

Keywords

  • ECoG
  • audiovisual
  • iEEG
  • intracranial
  • multisensory
  • sEEG
  • speech

ASJC Scopus subject areas

  • Neuroscience(all)

Fingerprint

Dive into the research topics of 'Visual speech differentially modulates beta, theta, and high gamma bands in auditory cortex'. Together they form a unique fingerprint.

Cite this