Predicting human gaze using low-level saliency combined with face detection

Moran Cerf*, Jonathan Harel, Wolfgang Einhäuser, Christof Koch

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

219 Scopus citations

Abstract

Under natural viewing conditions, human observers shift their gaze to allocate processing resources to subsets of the visual input. Many computational models try to predict such voluntary eye and attentional shifts. Although the important role of high level stimulus properties (e.g., semantic information) in search stands undisputed, most models are based on low-level image properties. We here demonstrate that a combined model of face detection and low-level saliency significantly outperforms a low-level model in predicting locations humans fixate on, based on eye-movement recordings of humans observing photographs of natural scenes, most of which contained at least one person. Observers, even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations; furthermore, they exhibit more similar scanpaths when faces are present. Remarkably, our model's predictive performance in images that do not contain faces is not impaired, and is even improved in some cases by spurious face detector responses.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference
StatePublished - Dec 1 2009
Event21st Annual Conference on Neural Information Processing Systems, NIPS 2007 - Vancouver, BC, Canada
Duration: Dec 3 2007Dec 6 2007

Other

Other21st Annual Conference on Neural Information Processing Systems, NIPS 2007
CountryCanada
CityVancouver, BC
Period12/3/0712/6/07

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Predicting human gaze using low-level saliency combined with face detection'. Together they form a unique fingerprint.

  • Cite this

    Cerf, M., Harel, J., Einhäuser, W., & Koch, C. (2009). Predicting human gaze using low-level saliency combined with face detection. In Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference