Predicting cancer outcomes from histology and genomics using convolutional networks

Pooya Mobadersany, Safoora Yousefi, Mohamed Amgad, David A. Gutman, Jill S. Barnholtz-Sloan, José E. Velázquez Vega, Daniel J. Brat, Lee A.D. Cooper*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

225 Scopus citations


Cancer histology reflects underlying molecular processes and disease progression and contains rich phenotypic information that is predictive of patient outcomes. In this study, we show a computational approach for learning patient outcomes from digital pathology images using deep learning to combine the power of adaptive machine learning algorithms with traditional survival models. We illustrate how these survival convolutional neural networks (SCNNs) can integrate information from both histology images and genomic biomarkers into a single unified framework to predict time-to-event outcomes and show prediction accuracy that surpasses the current clinical paradigm for predicting the overall survival of patients diagnosed with glioma. We use statistical sampling techniques to address challenges in learning survival from histology images, including tumor heterogeneity and the need for large training cohorts. We also provide insights into the prediction mechanisms of SCNNs, using heat map visualization to show that SCNNs recognize important structures, like microvascular proliferation, that are related to prognosis and that are used by pathologists in grading. These results highlight the emerging role of deep learning in precision medicine and suggest an expanding utility for computational analysis of histology in the future practice of pathology.

Original languageEnglish (US)
Pages (from-to)E2970-E2979
JournalProceedings of the National Academy of Sciences of the United States of America
Issue number13
StatePublished - Mar 27 2018


  • Artificial intelligence
  • Cancer
  • Deep learning
  • Digital pathology
  • Machine learning

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'Predicting cancer outcomes from histology and genomics using convolutional networks'. Together they form a unique fingerprint.

Cite this