Respond-CAM: Analyzing deep models for 3D imaging data by visualizations

Guannan Zhao, Bo Zhou, Kaiwen Wang, Rui Jiang, Min Xu*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

40 Scopus citations

Abstract

The convolutional neural network (CNN) has become a powerful tool for various biomedical image analysis tasks, but there is a lack of visual explanation for the machinery of CNNs. In this paper, we present a novel algorithm, Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based models interpretable by visualizing input regions that are important for predictions, especially for biomedical 3D imaging data inputs. Our method uses the gradients of any target concept (e.g. the score of target class) that flow into a convolutional layer. The weighted feature maps are combined to produce a heatmap that highlights the important regions in the image for predicting the target concept. We prove a preferable sum-to-score property of the Respond-CAM and verify its significant improvement on 3D images from the current state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D images show that Respond-CAM achieves superior performance on visualizing the CNNs with 3D biomedical image inputs, and is able to get reasonably good results on visualizing the CNNs with natural image inputs. The Respond-CAM is an efficient and reliable approach for visualizing the CNN machinery, and is applicable to a wide variety of CNN model families and image analysis tasks. Our code is available at: https://github.com/xulabs/projects/tree/master/respond_cam.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
EditorsJulia A. Schnabel, Christos Davatzikos, Carlos Alberola-López, Gabor Fichtinger, Alejandro F. Frangi
PublisherSpringer Verlag
Pages485-492
Number of pages8
ISBN (Print)9783030009274
DOIs
StatePublished - 2018
Event21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 20 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11070 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
Country/TerritorySpain
CityGranada
Period9/16/189/20/18

Funding

We thank Dr. Xiaodan Liang for suggestions. This work was supported in part by U.S. National Institutes of Health grant P41 GM103712. Min Xu acknowledges support from Samuel and Emma Winters Foundation. Rui Jiang is a RONG professor at the Institute for Data Science, Tsinghua University. Acknowledgements. We thank Dr. Xiaodan Liang for suggestions. This work was supported in part by U.S. National Institutes of Health grant P41 GM103712. Min Xu acknowledges support from Samuel and Emma Winters Foundation. Rui Jiang is a RONG professor at the Institute for Data Science, Tsinghua University.

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Respond-CAM: Analyzing deep models for 3D imaging data by visualizations'. Together they form a unique fingerprint.

Cite this