Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets

Stephanie A. Harmon; Thomas H. Sanford; Sheng Xu; Evrim B. Turkbey; Holger Roth; Ziyue Xu; Dong Yang; Andriy Myronenko; Victoria Anderson; Amel Amalou; Maxime Blain; Michael Kassin; Dilara Long; Nicole Varble; Stephanie M. Walker; Ulas Bagci; Anna Maria Ierardi; Elvira Stellato; Guido Giovanni Plensich; Giuseppe Franceschelli; Cristiano Girlando; Giovanni Irmici; Dominic Labella; Dima Hammoud; Ashkan Malayeri; Elizabeth Jones; Ronald M. Summers; Peter L. Choyke; Daguang Xu; Mona Flores; Kaku Tamura; Hirofumi Obinata; Hitoshi Mori; Francesca Patella; Maurizio Cariati; Gianpaolo Carrafiello; Peng An; Bradford J. Wood; Baris Turkbey

doi:10.1038/s41467-020-17971-2

Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets

Stephanie A. Harmon, Thomas H. Sanford, Sheng Xu, Evrim B. Turkbey, Holger Roth, Ziyue Xu, Dong Yang, Andriy Myronenko, Victoria Anderson, Amel Amalou, Maxime Blain, Michael Kassin, Dilara Long, Nicole Varble, Stephanie M. Walker, Ulas Bagci, Anna Maria Ierardi, Elvira Stellato, Guido Giovanni Plensich, Giuseppe FranceschelliCristiano Girlando, Giovanni Irmici, Dominic Labella, Dima Hammoud, Ashkan Malayeri, Elizabeth Jones, Ronald M. Summers, Peter L. Choyke, Daguang Xu, Mona Flores, Kaku Tamura, Hirofumi Obinata, Hitoshi Mori, Francesca Patella, Maurizio Cariati, Gianpaolo Carrafiello, Peng An, Bradford J. Wood^*, Baris Turkbey^*

^*Corresponding author for this work

Radiology

Research output: Contribution to journal › Article › peer-review

436 Scopus citations

Abstract

Chest CT is emerging as a valuable diagnostic tool for clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to aid in rapid evaluation of CT scans for differentiation of COVID-19 findings from other clinical entities. Here we show that a series of deep learning algorithms, trained in a diverse multinational cohort of 1280 patients to localize parietal pleura/lung parenchyma followed by classification of COVID-19 pneumonia, can achieve up to 90.8% accuracy, with 84% sensitivity and 93% specificity, as evaluated in an independent test set (not included in training and validation) of 1337 patients. Normal controls included chest CTs from oncology, emergency, and pneumonia-related indications. The false positive rate in 140 patients with laboratory confirmed other (non COVID-19) pneumonias was 10%. AI-based algorithms can readily identify CT scans with COVID-19 associated pneumonia, as well as distinguish non-COVID related pneumonias with high specificity in diverse patient populations.

Original language	English (US)
Article number	4080
Journal	Nature communications
Volume	11
Issue number	1
DOIs	https://doi.org/10.1038/s41467-020-17971-2
State	Published - Dec 1 2020

Funding

This work was supported by the NIH Center for Interventional Oncology and the Intramural Research Program of the National Institutes of Health (NIH) by intramural NIH Grants NIH Z01 1ZID # BC011242 and CL040015 and the NIH Intramural Targeted Anti-COVID-19 (ITAC) Program. This project has been funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health, under Contract No. 75N91019D00024, Task Order No. 75N91019F00129. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. NIH may have intellectual property in the field. The opinions expressed herein are those of the authors alone, and do necessarily represent those of the NIH nor the US Government. Mention of commercial products or policies should not be misconstrued as endorsement by the NIH. NIH and NVIDIA have a Cooperative Research and Development Agreement. M.B. is a recipient of the 2019 Alain Rahmouni SFR-CERF research grant provided by the French Society of Radiology together with the French Academic College of Radiology. We are thankful to Cinzia Mennini, MD, contributed toward data and discussions for this work. Thanks also for assistance in discussions and guidance: William Dahut, Tom Mistelli, John Gallin, Bruce Tromberg, Cliff Lane, Ken Rose, Jeff Solomon, Irwin Feuerstein, David Spiro, Kaiyong Sun, Rob Suh, Hayet Amalou, Corey Arnold, Dieter Enzmann, Steve Raman, Gregg Cohen, Andrew Feng, Abdul Hamid-Halabi, Kimberly Powell, Wentau Zhu, Xiaosong Wang, Jeff Plum, and Colleen Ruan.

ASJC Scopus subject areas

General Chemistry
General Biochemistry, Genetics and Molecular Biology
General Physics and Astronomy

Access to Document

10.1038/s41467-020-17971-2

Cite this

Harmon, S. A., Sanford, T. H., Xu, S., Turkbey, E. B., Roth, H., Xu, Z., Yang, D., Myronenko, A., Anderson, V., Amalou, A., Blain, M., Kassin, M., Long, D., Varble, N., Walker, S. M., Bagci, U., Ierardi, A. M., Stellato, E., Plensich, G. G., ... Turkbey, B. (2020). Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets. Nature communications, 11(1), Article 4080. https://doi.org/10.1038/s41467-020-17971-2

@article{404fe91c4de54901a5876126baf55262,

title = "Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets",

abstract = "Chest CT is emerging as a valuable diagnostic tool for clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to aid in rapid evaluation of CT scans for differentiation of COVID-19 findings from other clinical entities. Here we show that a series of deep learning algorithms, trained in a diverse multinational cohort of 1280 patients to localize parietal pleura/lung parenchyma followed by classification of COVID-19 pneumonia, can achieve up to 90.8% accuracy, with 84% sensitivity and 93% specificity, as evaluated in an independent test set (not included in training and validation) of 1337 patients. Normal controls included chest CTs from oncology, emergency, and pneumonia-related indications. The false positive rate in 140 patients with laboratory confirmed other (non COVID-19) pneumonias was 10%. AI-based algorithms can readily identify CT scans with COVID-19 associated pneumonia, as well as distinguish non-COVID related pneumonias with high specificity in diverse patient populations.",

author = "Harmon, {Stephanie A.} and Sanford, {Thomas H.} and Sheng Xu and Turkbey, {Evrim B.} and Holger Roth and Ziyue Xu and Dong Yang and Andriy Myronenko and Victoria Anderson and Amel Amalou and Maxime Blain and Michael Kassin and Dilara Long and Nicole Varble and Walker, {Stephanie M.} and Ulas Bagci and Ierardi, {Anna Maria} and Elvira Stellato and Plensich, {Guido Giovanni} and Giuseppe Franceschelli and Cristiano Girlando and Giovanni Irmici and Dominic Labella and Dima Hammoud and Ashkan Malayeri and Elizabeth Jones and Summers, {Ronald M.} and Choyke, {Peter L.} and Daguang Xu and Mona Flores and Kaku Tamura and Hirofumi Obinata and Hitoshi Mori and Francesca Patella and Maurizio Cariati and Gianpaolo Carrafiello and Peng An and Wood, {Bradford J.} and Baris Turkbey",

note = "Publisher Copyright: {\textcopyright} 2020, The Author(s).",

year = "2020",

month = dec,

day = "1",

doi = "10.1038/s41467-020-17971-2",

language = "English (US)",

volume = "11",

journal = "Nature communications",

issn = "2041-1723",

publisher = "Nature Research",

number = "1",

}

Harmon, SA, Sanford, TH, Xu, S, Turkbey, EB, Roth, H, Xu, Z, Yang, D, Myronenko, A, Anderson, V, Amalou, A, Blain, M, Kassin, M, Long, D, Varble, N, Walker, SM, Bagci, U, Ierardi, AM, Stellato, E, Plensich, GG, Franceschelli, G, Girlando, C, Irmici, G, Labella, D, Hammoud, D, Malayeri, A, Jones, E, Summers, RM, Choyke, PL, Xu, D, Flores, M, Tamura, K, Obinata, H, Mori, H, Patella, F, Cariati, M, Carrafiello, G, An, P, Wood, BJ & Turkbey, B 2020, 'Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets', Nature communications, vol. 11, no. 1, 4080. https://doi.org/10.1038/s41467-020-17971-2

TY - JOUR

T1 - Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets

AU - Harmon, Stephanie A.

AU - Sanford, Thomas H.

AU - Xu, Sheng

AU - Turkbey, Evrim B.

AU - Roth, Holger

AU - Xu, Ziyue

AU - Yang, Dong

AU - Myronenko, Andriy

AU - Anderson, Victoria

AU - Amalou, Amel

AU - Blain, Maxime

AU - Kassin, Michael

AU - Long, Dilara

AU - Varble, Nicole

AU - Walker, Stephanie M.

AU - Bagci, Ulas

AU - Ierardi, Anna Maria

AU - Stellato, Elvira

AU - Plensich, Guido Giovanni

AU - Franceschelli, Giuseppe

AU - Girlando, Cristiano

AU - Irmici, Giovanni

AU - Labella, Dominic

AU - Hammoud, Dima

AU - Malayeri, Ashkan

AU - Jones, Elizabeth

AU - Summers, Ronald M.

AU - Choyke, Peter L.

AU - Xu, Daguang

AU - Flores, Mona

AU - Tamura, Kaku

AU - Obinata, Hirofumi

AU - Mori, Hitoshi

AU - Patella, Francesca

AU - Cariati, Maurizio

AU - Carrafiello, Gianpaolo

AU - An, Peng

AU - Wood, Bradford J.

AU - Turkbey, Baris

PY - 2020/12/1

Y1 - 2020/12/1

N2 - Chest CT is emerging as a valuable diagnostic tool for clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to aid in rapid evaluation of CT scans for differentiation of COVID-19 findings from other clinical entities. Here we show that a series of deep learning algorithms, trained in a diverse multinational cohort of 1280 patients to localize parietal pleura/lung parenchyma followed by classification of COVID-19 pneumonia, can achieve up to 90.8% accuracy, with 84% sensitivity and 93% specificity, as evaluated in an independent test set (not included in training and validation) of 1337 patients. Normal controls included chest CTs from oncology, emergency, and pneumonia-related indications. The false positive rate in 140 patients with laboratory confirmed other (non COVID-19) pneumonias was 10%. AI-based algorithms can readily identify CT scans with COVID-19 associated pneumonia, as well as distinguish non-COVID related pneumonias with high specificity in diverse patient populations.

AB - Chest CT is emerging as a valuable diagnostic tool for clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to aid in rapid evaluation of CT scans for differentiation of COVID-19 findings from other clinical entities. Here we show that a series of deep learning algorithms, trained in a diverse multinational cohort of 1280 patients to localize parietal pleura/lung parenchyma followed by classification of COVID-19 pneumonia, can achieve up to 90.8% accuracy, with 84% sensitivity and 93% specificity, as evaluated in an independent test set (not included in training and validation) of 1337 patients. Normal controls included chest CTs from oncology, emergency, and pneumonia-related indications. The false positive rate in 140 patients with laboratory confirmed other (non COVID-19) pneumonias was 10%. AI-based algorithms can readily identify CT scans with COVID-19 associated pneumonia, as well as distinguish non-COVID related pneumonias with high specificity in diverse patient populations.

UR - http://www.scopus.com/inward/record.url?scp=85089409297&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85089409297&partnerID=8YFLogxK

U2 - 10.1038/s41467-020-17971-2

DO - 10.1038/s41467-020-17971-2

M3 - Article

C2 - 32796848

AN - SCOPUS:85089409297

SN - 2041-1723

VL - 11

JO - Nature communications

JF - Nature communications

IS - 1

M1 - 4080

ER -

Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets

Abstract

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this