Audiovisual sentence recognition not predicted by susceptibility to the McGurk effect

Kristin J. Van Engen, Zilong Xie, Bharath Chandrasekaran*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

45 Scopus citations

Abstract

In noisy situations, visual information plays a critical role in the success of speech communication: listeners are better able to understand speech when they can see the speaker. Visual influence on auditory speech perception is also observed in the McGurk effect, in which discrepant visual information alters listeners’ auditory perception of a spoken syllable. When hearing /ba/ while seeing a person saying /ga/, for example, listeners may report hearing /da/. Because these two phenomena have been assumed to arise from a common integration mechanism, the McGurk effect has often been used as a measure of audiovisual integration in speech perception. In this study, we test whether this assumed relationship exists within individual listeners. We measured participants’ susceptibility to the McGurk illusion as well as their ability to identify sentences in noise across a range of signal-to-noise ratios in audio-only and audiovisual modalities. Our results do not show a relationship between listeners’ McGurk susceptibility and their ability to use visual cues to understand spoken sentences in noise, suggesting that McGurk susceptibility may not be a valid measure of audiovisual integration in everyday speech processing.

Original languageEnglish (US)
Pages (from-to)396-403
Number of pages8
JournalAttention, Perception, and Psychophysics
Volume79
Issue number2
DOIs
StatePublished - Feb 1 2017

Funding

This work was supported by NIH-NIDCD grant R01DC013315 to BC. We thank the SoundBrain laboratory research assistants, especially Jacie Richardson and Cat Han, for data collection and scoring. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Keywords

  • Audiovisual speech perception
  • McGurk effect
  • Speech perception in noise

ASJC Scopus subject areas

  • Language and Linguistics
  • Experimental and Cognitive Psychology
  • Sensory Systems
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Audiovisual sentence recognition not predicted by susceptibility to the McGurk effect'. Together they form a unique fingerprint.

Cite this