Exploring behavior representation for learning analytics

Marcelo Worsley, Stefan Scherer, Louis Philippe Morency, Paulo Blikstein

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations


Multimodal analysis has long been an integral part of studying learning. Historically multimodal analyses of learning have been extremely laborious and time intensive. However, researchers have recently been exploring ways to use multimodal computational analysis in the service of studying how people learn in complex learning environments. In an effort to advance this research agenda, we present a comparative analysis of four different data segmentation techniques. In particular, we propose affect- and pose-based data segmentation, as alternatives to human-based segmentation, and fixed-window segmentation. In a study of ten dyads working on an open-ended engineering design task, we find that affect- and pose-based segmentation are more effective, than traditional approaches, for drawing correlations between learning-relevant constructs, and multimodal behaviors. We also find that pose-based segmentation outperforms the two more traditional segmentation strategies for predicting student success on the hands-on task. In this paper we discuss the algorithms used, our results, and the implications that this work may have in non-education-related contexts.

Original languageEnglish (US)
Title of host publicationICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery, Inc
Number of pages8
ISBN (Electronic)9781450339124
StatePublished - Nov 9 2015
EventACM International Conference on Multimodal Interaction, ICMI 2015 - Seattle, United States
Duration: Nov 9 2015Nov 13 2015

Publication series

NameICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction


OtherACM International Conference on Multimodal Interaction, ICMI 2015
Country/TerritoryUnited States


  • Interaction analysis
  • Learning sciences
  • Modeling

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Human-Computer Interaction


Dive into the research topics of 'Exploring behavior representation for learning analytics'. Together they form a unique fingerprint.

Cite this