Gaze2Segment: A pilot study for integrating eye-tracking technology into medical image segmentation

Naji Khosravan, Haydar Celik, Baris Turkbey, Ruida Cheng, Evan McCreedy, Matthew McAuliffe, Sandra Bednarova, Elizabeth Jones, Xinjian Chen, Peter Choyke, Bradford Wood, Ulas Bagci*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Scopus citations

Abstract

In this study, we developed a novel system, called Gaze2Segment, integrating biological and computer vision techniques to support radiologists’ reading experience with an automatic image segmentation task. During diagnostic assessment of lung CT scans, the radiologists’ gaze information were used to create a visual attention map. Next, this map was combined with a computer-derived saliency map, extracted from the gray-scale CT images. The visual attention map was used as an input for indicating roughly the location of a region of interest.With computer-derived saliency information, on the other hand, we aimed at finding foreground and background cues for the object of interest found in the previous step. These cues are used to initiate a seed-based delineation process. The proposed Gaze2Segment achieved a dice similarity coefficient of 86% and Hausdorff distance of 1.45 mm as a segmentation accuracy. To the best of our knowledge, Gaze2Segment is the first true integration of eye-tracking technology into a medical image segmentation task without the need for any further user-interaction.

Original languageEnglish (US)
Title of host publicationMedical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging - MICCAI 2016 International Workshops, MCV and BAMBI, Revised Selected Papers
EditorsTal Arbel, Georg Langs, Mark Jenkinson, Bjoern Menze, William M. Wells III, Albert C.S. Chung, B. Michael Kelm, Weidong Cai, Albert Montillo, Dimitris Metaxas, M. Jorge Cardoso, Shaoting Zhang, Annemie Ribbens, Henning Muller
PublisherSpringer Verlag
Pages94-104
Number of pages11
ISBN (Print)9783319611877
DOIs
StatePublished - 2017
EventInternational Workshop on Medical Computer Vision, MCV 2016, and of the International Workshop on Bayesian and grAphical Models for Biomedical Imaging, BAMBI 2016, held in conjunction with the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016 - Athens, Greece
Duration: Oct 21 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10081 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Workshop on Medical Computer Vision, MCV 2016, and of the International Workshop on Bayesian and grAphical Models for Biomedical Imaging, BAMBI 2016, held in conjunction with the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016
Country/TerritoryGreece
CityAthens
Period10/21/1610/21/16

Keywords

  • Eye tracking
  • Human computer interface
  • Local saliency
  • Medical image segmentation
  • Visual attention

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Gaze2Segment: A pilot study for integrating eye-tracking technology into medical image segmentation'. Together they form a unique fingerprint.

Cite this