Sounds exaggerate visual shape

Timothy D. Sweeny*, Emmanuel Guzman-Martinez, Laura Ortega, Marcia F Grabowecky, Satoru Suzuki

*Corresponding author for this work

Research output: Contribution to journalArticle

23 Scopus citations

Abstract

While perceiving speech, people see mouth shapes that are systematically associated with sounds. In particular, a vertically stretched mouth produces a /woo/ sound, whereas a horizontally stretched mouth produces a /wee/ sound. We demonstrate that hearing these speech sounds alters how we see aspect ratio, a basic visual feature that contributes to perception of 3D space, objects and faces. Hearing a /woo/ sound increases the apparent vertical elongation of a shape, whereas hearing a /wee/ sound increases the apparent horizontal elongation. We further demonstrate that these sounds influence aspect ratio coding. Viewing and adapting to a tall (or flat) shape makes a subsequently presented symmetric shape appear flat (or tall). These aspect ratio aftereffects are enhanced when associated speech sounds are presented during the adaptation period, suggesting that the sounds influence visual population coding of aspect ratio. Taken together, these results extend previous demonstrations that visual information constrains auditory perception by showing the converse - speech sounds influence visual perception of a basic geometric feature.

Original languageEnglish (US)
Pages (from-to)194-200
Number of pages7
JournalCognition
Volume124
Issue number2
DOIs
StatePublished - Aug 1 2012

Keywords

  • Aspect ratio
  • Auditory-visual
  • Crossmodal
  • Shape perception
  • Speech perception

ASJC Scopus subject areas

  • Language and Linguistics
  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Linguistics and Language
  • Cognitive Neuroscience

Fingerprint Dive into the research topics of 'Sounds exaggerate visual shape'. Together they form a unique fingerprint.

  • Cite this