Detecting Screen Presence with Activity-Oriented RGB Camera in Egocentric Videos

Amit Adate, Soroush Shahi, Rawan Alharbi, Sougata Sen, Yang Gao, Aggelos K. Katsaggelos, Nabil Alshurafa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Screen time is associated with several health risk behaviors including mindless eating, sedentary behavior, and decreased academic performance. Screen time behavior is traditionally assessed with self-report measures, which are known to be burdensome, inaccurate, and imprecise. Recent methods to automatically detect screen time are geared more towards detecting television screens from wearable cameras that record high-resolution video. Activity-oriented wearable cameras (i.e., cameras oriented towards the wearer with a fisheye lens) have recently been designed and shown to reduce privacy concerns, yet pose a greater challenge in capturing screens due to their orientation and fewer pixels on target. Methods that detect screens from low-power, low-resolution wearable camera video are needed given the increased adoption of such devices in longitudinal studies. We propose a method that leverages deep learning algorithms and lower-resolution images from an activity-oriented camera to detect screen presence from multiple types of screens with high variability of pixel on target (e.g., near and far TV, smartphones, laptops, and tablets). We test our system in a real-world study comprising 10 individuals, 80 hours of data, and 1.2 million low-resolution RGB frames. Our results outperform existing state-of-the-art video screen detection methods yielding an F1-score of 81%. This paper demonstrates the potential for detecting screen-watching behavior in longitudinal studies using activity-oriented cameras, paving the way for a nuanced understanding of screen time's relationship with health risk behaviors.

Original languageEnglish (US)
Title of host publication2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages403-408
Number of pages6
ISBN (Electronic)9781665416474
DOIs
StatePublished - 2022
Event2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022 - Pisa, Italy
Duration: Mar 21 2022Mar 25 2022

Publication series

Name2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022

Conference

Conference2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022
Country/TerritoryItaly
CityPisa
Period3/21/223/25/22

Keywords

  • Egocentric Videos
  • Fisheye Lens
  • Object Detection
  • Wearable Camera

ASJC Scopus subject areas

  • Artificial Intelligence
  • Modeling and Simulation
  • Health Informatics
  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Detecting Screen Presence with Activity-Oriented RGB Camera in Egocentric Videos'. Together they form a unique fingerprint.

Cite this