Cans and cants: Computational potentials for multimodality with a case study in head position

Rob Voigt*, Penelope Eckert, Dan Jurafsky, Robert J. Podesva

*Corresponding author for this work

Research output: Contribution to journalArticle

3 Scopus citations

Abstract

As the study of embodiment and multimodality in interaction grows in importance, there is a need for novel methodological approaches to understand how multimodal variables pattern together along social and contextual lines, and how they systematically coalesce in communicative meanings. In this work, we propose to adopt computational tools to generate replicable annotations of bodily variables: these can be examined statistically to understand their patterning with other variables across diverse speakers and interactional contexts, and can help organize qualitative analyses of large datasets. We demonstrate the possibilities thereby with a case study in head cant (side-to-side tilt of the head) in a dataset of video blogs and laboratory-collected interactions, computationally extracting cant and prosody from video and audio and analyzing their interactions, looking at gender in particular. We find that head cant indexes an orientation towards the interlocutor and a sense of shared understanding, can serve a ‘bracketing’ function in interaction (for speakers to create parentheticals or asides), and has gendered associations with prosodic markers and interactional discourse particles.

Original languageEnglish (US)
Pages (from-to)677-711
Number of pages35
JournalJournal of Sociolinguistics
Volume20
Issue number5
DOIs
StatePublished - Nov 1 2016

Keywords

  • body positioning
  • computer vision
  • Embodiment
  • gender
  • head cant
  • interaction
  • multimodality
  • prosody

ASJC Scopus subject areas

  • Language and Linguistics
  • Sociology and Political Science
  • Philosophy
  • Linguistics and Language
  • History and Philosophy of Science

Fingerprint Dive into the research topics of 'Cans and cants: Computational potentials for multimodality with a case study in head position'. Together they form a unique fingerprint.

  • Cite this