Speaker movement correlates with prosodic indicators of engagement

Rob Voigt, Robert J. Podesva, Dan Jurafsky

Research output: Contribution to journalConference articlepeer-review

8 Scopus citations

Abstract

Recent research on multimodal prosody has begun to identify associations between discrete body movements and categorical acoustic prosodic events such as pitch accents and boundaries. We propose to generalize this work to understand more about continuous prosodic phenomena distributed over a phrase -like those indicative of speaker engagement -and how they covary with bodily movements. We introduce movement amplitude, a new vision-based metric for estimating continuous body movements over time from video by quantifying frameto-frame visual changes. Application of this automatic metric to a collection of video monologues demonstrates that speakers move more during phrases in which their pitch and intensity are higher and more variable. These findings offer further evidence for the relationship between acoustic and visual prosody, and suggest a previously unreported quantitative connection between raw bodily movement and speaker engagement.

Original languageEnglish (US)
Pages (from-to)70-74
Number of pages5
JournalProceedings of the International Conference on Speech Prosody
DOIs
StatePublished - Jan 1 2014
Event7th International Conference on Speech Prosody, SP 2014 - Dublin, Ireland
Duration: May 20 2014May 23 2014

Keywords

  • Acoustic prosody
  • Automatic methods
  • Gesture
  • Movement
  • Speech-gesture interface
  • Visual prosody

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Fingerprint Dive into the research topics of 'Speaker movement correlates with prosodic indicators of engagement'. Together they form a unique fingerprint.

Cite this