SPIDERS+: A light-weight, wireless, and low-cost glasses-based wearable platform for emotion sensing and bio-signal acquisition

Jingping Nie, Yanchen Liu, Yigong Hu, Yuanyuting Wang, Stephen Xia, Matthias Preindl, Xiaofan Jiang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

We present a System for Processing In-situ Bio-signal Data for Emotion Recog-nition and Sensing (SPIDERS+) – a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user's facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions to detect eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements, using the bio-signals acquired from three non-contact sensors (IR camera, proximity sensor, and IMU). SPIDERS+ distinguishes between different classes of apparent and real emotion states based on the aforementioned four bio-signals. We prototype advanced functionalities including facial expression detection and real emotion classification with a facial expression detector based on landmarks and optical flow that leverages changes in a user's eyebrows and eye shapes to achieve the average true positive rate up to 83.87%, as well as a pupillometry-based real emotion classifier with higher accuracy than existing low-cost wearable platforms that use sensors requiring skin contact. In addition, SPIDERS+ also includes optional contact-based sensors such as photoplethysmography (PPG) and multi-channel electroencephalogram (EEG) to monitor other bio-signals. SPIDERS+ costs less than $20 to assemble and can continuously run for up to 9 h before recharging. We design, implement and evaluate accompanying desktop and smartphone applications to demonstrate that SPIDERS+ is a truly wireless and portable platform that can impact a wide range of applications, where the knowledge of the user's emotional state is critical.

Original languageEnglish (US)
Article number101424
JournalPervasive and Mobile Computing
Volume75
DOIs
StatePublished - Aug 2021

Funding

This research was partially supported by the National Science Foundation under Grant Numbers CNS-1704899 , CNS-1815274 , CNS-11943396 , and CNS-1837022 . The views and conclusions contained here are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Columbia University, NSF, or the U.S. Government or any of its agencies.

Keywords

  • Bio-signal acquisition
  • Emotion monitoring
  • Facial expression detection
  • Pupillometry
  • Wearables

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'SPIDERS+: A light-weight, wireless, and low-cost glasses-based wearable platform for emotion sensing and bio-signal acquisition'. Together they form a unique fingerprint.

Cite this