Computationally Augmented Ethnography: Emotion Tracking and Learning in Museum Games

Kit Martin*, Emily Q. Wang, Connor Bain, Marcelo Worsley

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

In this paper, we describe a way of using multi-modal learning analytics to augment qualitative data. We extract facial expressions that may indicate particular emotions from videos of dyads playing an interactive table-top game built for a museum. From this data, we explore the correlation between students’ understanding of the biological and complex systems concepts showcased in the learning environment and their facial expressions. First, we show how information retrieval techniques can be used on facial expression features to investigate emotional variation during key moments of the interaction. Second, we connect these features to moments of learning identified by traditional qualitative methods. Finally, we present an initial pilot using these methods in concert to identify key moments in multiple modalities. We end with a discussion of our preliminary findings on interweaving machine and human analytical approaches.

Original languageEnglish (US)
Title of host publicationAdvances in Quantitative Ethnography - 1st International Conference, ICQE 2019, Proceedings
EditorsBrendan Eagan, Amanda Siebert-Evenstone, Morten Misfeldt
PublisherSpringer
Pages141-153
Number of pages13
ISBN (Print)9783030332310
DOIs
StatePublished - Jan 1 2019
Event1st International Conference on Quantitative Ethnography, ICQE 2019 - Madison, United States
Duration: Oct 20 2019Oct 22 2019

Publication series

NameCommunications in Computer and Information Science
Volume1112
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

Conference1st International Conference on Quantitative Ethnography, ICQE 2019
CountryUnited States
CityMadison
Period10/20/1910/22/19

Keywords

  • Affect tracking
  • Game-based learning
  • Multimodal learning analytics
  • Physical traces

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)

Fingerprint Dive into the research topics of 'Computationally Augmented Ethnography: Emotion Tracking and Learning in Museum Games'. Together they form a unique fingerprint.

Cite this