Primary and multisensory cortical activity is correlated with audiovisual percepts

Margo McKenna Benoit, Tommi Raij, Fa Hsuan Lin, Iiro P. Jääskeläinen, Steven Stufflebeam

Research output: Contribution to journalArticlepeer-review

53 Scopus citations

Abstract

Incongruent auditory and visual stimuli can elicit audiovisual illusions such as the McGurk effect where visual /ka/ and auditory /pa/ fuse into another percept such as/ta/. In the present study, human brain activity was measured with adaptation functional magnetic resonance imaging to investigate which brain areas support such audiovisual illusions. Subjects viewed trains of four movies beginning with three congruent /pa/ stimuli to induce adaptation. The fourth stimulus could be (i) another congruent /pa/, (ii) a congruent /ka/, (iii) an incongruent stimulus that evokes the McGurk effect in susceptible individuals (lips /ka/ voice /pa/), or (iv) the converse combination that does not cause the McGurk effect (lips /pa/ voice/ ka/). This paradigm was predicted to show increased release from adaptation (i.e. stronger brain activation) when the fourth movie and the related percept was increasingly different from the three previous movies. A stimulus change in either the auditory or the visual stimulus from /pa/ to /ka/ (iii, iv) produced within-modality and cross-modal responses in primary auditory and visual areas. A greater release from adaptation was observed for incongruent non-McGurk (iv) compared to incongruent McGurk (iii) trials. A network including the primary auditory and visual cortices, nonprimary auditory cortex, and several multisensory areas (superior temporal sulcus, intraparietal sulcus, insula, and pre-central cortex) showed a correlation between perceiving the McGurk effect and the fMRI signal, suggesting that these areas support the audiovisual illusion.

Original languageEnglish (US)
Pages (from-to)526-538
Number of pages13
JournalHuman Brain Mapping
Volume31
Issue number4
DOIs
StatePublished - Apr 2010

Keywords

  • Auditory cortex
  • Cochlear implant
  • Hearing
  • Speech perception
  • Visual cortex
  • fMRI

ASJC Scopus subject areas

  • Anatomy
  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Clinical Neurology

Fingerprint Dive into the research topics of 'Primary and multisensory cortical activity is correlated with audiovisual percepts'. Together they form a unique fingerprint.

Cite this