Auditory rhythms are systemically associated with spatial-frequency and density information in visual scenes

Aleksandra Sherman*, Marcia F Grabowecky, Satoru Suzuki

*Corresponding author for this work

Research output: Contribution to journalArticle

4 Scopus citations

Abstract

A variety of perceptual correspondences between auditory and visual features have been reported, but few studies have investigated how rhythm, an auditory feature defined purely by dynamics relevant to speech and music, interacts with visual features. Here, we demonstrate a novel crossmodal association between auditory rhythm and visual clutter. Participants were shown a variety of visual scenes from diverse categories and asked to report the auditory rhythm that perceptually matched each scene by adjusting the rate of amplitude modulation (AM) of a sound. Participants matched each scene to a specific AM rate with surprising consistency. A spatial-frequency analysis showed that scenes with greater contrast energy in midrange spatial frequencies were matched to faster AM rates. Bandpass-filtering the scenes indicated that greater contrast energy in this spatial-frequency range was associated with an abundance of object boundaries and contours, suggesting that participants matched more cluttered scenes to faster AM rates. Consistent with this hypothesis, AM-rate matches were strongly correlated with perceived clutter. Additional results indicated that both AM-rate matches and perceived clutter depend on object-based (cycles per object) rather than retinal (cycles per degree of visual angle) spatial frequency. Taken together, these results suggest a systematic crossmodal association between auditory rhythm, representing density in the temporal domain, and visual clutter, representing object-based density in the spatial domain. This association may allow for the use of auditory rhythm to influence how visual clutter is perceived and attended.

Original languageEnglish (US)
Pages (from-to)740-746
Number of pages7
JournalPsychonomic Bulletin and Review
Volume20
Issue number4
DOIs
StatePublished - Aug 1 2013

Keywords

  • Amplitude modulation rate
  • Crossmodal
  • Density
  • Multisensory integration
  • Natural scenes
  • Spatial frequency
  • Visual clutter

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)

Fingerprint Dive into the research topics of 'Auditory rhythms are systemically associated with spatial-frequency and density information in visual scenes'. Together they form a unique fingerprint.

  • Cite this