SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry fromWearable Sensors

Yun C. Zhang, Shibo Zhang, Miao Liu, Elyse Daly, Samuel Battalio, Santosh Kumar, Bonnie Spring, James M. Rehg, Nabil Alshurafa

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

The development and validation of computational models to detect daily human behaviors (e.g., eating, smoking, brushing) using wearable devices requires labeled data collected from the natural field environment, with tight time synchronization of the micro-behaviors (e.g., start/end times of hand-to-mouth gestures during a smoking puff or an eating gesture) and the associated labels. Video data is increasingly being used for such label collection. Unfortunately, wearable devices and video cameras with independent (and drifting) clocks make tight time synchronization challenging. To address this issue, we present the Window Induced Shift Estimation method for Synchronization (SyncWISE) approach. We demonstrate the feasibility and effectiveness of our method by synchronizing the timestamps of a wearable camera and wearable accelerometer from 163 videos representing 45.2 hours of data from 21 participants enrolled in a real-world smoking cessation study. Our approach shows significant improvement over the state-of-the-art, even in the presence of high data loss, achieving 90% synchronization accuracy given a synchronization tolerance of 700 milliseconds. Our method also achieves state-of-the-art synchronization performance on the CMU-MMAC dataset.

Original languageEnglish (US)
Article number107
JournalProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume4
Issue number3
DOIs
StatePublished - Sep 4 2020

Keywords

  • Accelerometry
  • Automatic Synchronization
  • Temporal Drift
  • Time Synchronization
  • Video
  • Wearable Camera
  • Wearable Sensor

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry fromWearable Sensors'. Together they form a unique fingerprint.

Cite this