Using semantic content as cues for better scanpath prediction

Moran Cerf*, E. Paxon Frady, Christof Koch

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

Under natural viewing conditions, human observers use shifts in gaze to allocate processing resources to subsets of the visual input. There are many computational models that try to predict these shifts in eye movement and attention. Although the important role of high level stimulus properties (e.g., semantic information) stands undisputed, most models are based solely on low-level image properties. We here demonstrate that a combined model of high-level object detection and low-level saliency significantly outperforms a low-level saliency model in predicting locations humans fixate on. The data is based on eye-movement recordings of humans observing photographs of natural scenes, which contained one of the following high-level stimuli: faces, text, scrambled text or cell phones. We show that observers - even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations, on text and scrambled text with a probability of over 65.1% and 57.9% respectively, and on cell phones with probability of 8.3%. This suggests that content with meaningful semantic information is significantly more likely to be seen earlier. Adding regions of interest (ROI), which depict the locations of the high-level meaningful features, significantly improves the prediction of a saliency model for stimuli with high semantic importance, while it has little effect for an object with no semantic meaning.

Original languageEnglish (US)
Title of host publicationProceedings of the Eye Tracking Research and Applications Symposium, ETRA 2008
Pages143-146
Number of pages4
DOIs
StatePublished - Dec 1 2008
EventEye Tracking Research and Applications Symposium, ETRA 2008 - Savannah, GA, United States
Duration: Mar 26 2008Mar 28 2008

Publication series

NameEye Tracking Research and Applications Symposium (ETRA)

Other

OtherEye Tracking Research and Applications Symposium, ETRA 2008
CountryUnited States
CitySavannah, GA
Period3/26/083/28/08

Keywords

  • Eye tracking
  • Natural scenes
  • Psychophysics

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Ophthalmology
  • Sensory Systems

Fingerprint Dive into the research topics of 'Using semantic content as cues for better scanpath prediction'. Together they form a unique fingerprint.

  • Cite this

    Cerf, M., Frady, E. P., & Koch, C. (2008). Using semantic content as cues for better scanpath prediction. In Proceedings of the Eye Tracking Research and Applications Symposium, ETRA 2008 (pp. 143-146). (Eye Tracking Research and Applications Symposium (ETRA)). https://doi.org/10.1145/1344471.1344508