A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics

Nati Daniel; Ariel Larey; Eliel Aknin; Garrett A. Osswald; Julie M. Caldwell; Mark Rochman; Margaret H. Collins; Guang Yu Yang; Nicoleta C. Arva; Kelley E. Capocelli; Marc E. Rothenberg; Yonatan Savir

doi:10.1109/EMBC48229.2022.9871086

A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics

Nati Daniel, Ariel Larey, Eliel Aknin, Garrett A. Osswald, Julie M. Caldwell, Mark Rochman, Margaret H. Collins, Guang Yu Yang, Nicoleta C. Arva, Kelley E. Capocelli, Marc E. Rothenberg, Yonatan Savir^*

^*Corresponding author for this work

Pathology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

14 Scopus citations

Abstract

Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.

Original language	English (US)
Title of host publication	44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3211-3217
Number of pages	7
ISBN (Electronic)	9781728127828
DOIs	https://doi.org/10.1109/EMBC48229.2022.9871086
State	Published - 2022
Event	44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022 - Glasgow, United Kingdom Duration: Jul 11 2022 → Jul 15 2022

Publication series

Name	Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume	2022-July
ISSN (Print)	1557-170X

Conference

Conference	44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
Country/Territory	United Kingdom
City	Glasgow
Period	7/11/22 → 7/15/22

Funding

This work was supported by Israel Science Foundation #1619/20, Rappa-port Foundation, Prince Center for Neurodegenerative Disorders, 3828931, NIH R01 AI045898-21, the CURED Foundation, and Dave and Denise Bunning Sunshine Foundation. CEGIR (U54 AI117804) is part of the Rare Disease Clinical Research Network, and is funded by ORDR, NIAID, NIDDK, NCATS, the American Partnership for Eosinophilic Disorders, CURED, Eosinophilic Family Coalition, and RDCRN DMCC (U2CTR002818). ζContributed equally to this work. ∗Corresponding author, [email protected] 1N.D. and Y.S. are with the Dept. of Physiology, Biophysics and System Biology, Faculty of Medicine, Technion, Israel. 2A.L. is with the Faculty of Computer Science, Technion, Israel. 3E.A. is with the Faculty of Industrial Engineering, Technion, Israel. 4G.A.O., 4J.M.C., 4M.R., and 4M.E.R. are with the Division of Allergy and Immunology, Cincinnati Children’s Hospital Medical Center, Dept. of Pediatrics, University of Cincinnati College of Medicine, OH, USA., 5M.H.C. is with the Dept. of Pathology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, USA. 6G.Y.Y. and 6N.C.A. are with the Dept. of Pathology, Ann & Robert H. Lurie Children’s Hospital of Chicago, Northwestern University, The Feinberg School of Medicine, Chicago, IL, USA. 7K.E.C. is with the Dept. of Pathology, Children’s Hospital Colorado, Aurora, CO, USA. This work was supported by Israel Science Foundation #1619/20, Rappaport Foundation, Prince Center for Neurodegenerative Disorders, 3828931, NIH R01 AI045898-21, the CURED Foundation, and Dave and Denise Bunning Sunshine Foundation. CEGIR (U54 AI117804).

Keywords

Decision support system
deep learning
digital pathology
eosinophilic esophagitis
whole slide image segmentation and classification

ASJC Scopus subject areas

Signal Processing
Biomedical Engineering
Computer Vision and Pattern Recognition
Health Informatics

Access to Document

10.1109/EMBC48229.2022.9871086

Cite this

Daniel, N., Larey, A., Aknin, E., Osswald, G. A., Caldwell, J. M., Rochman, M., Collins, M. H., Yang, G. Y., Arva, N. C., Capocelli, K. E., Rothenberg, M. E., & Savir, Y. (2022). A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. In 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022 (pp. 3211-3217). (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; Vol. 2022-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EMBC48229.2022.9871086

Daniel, Nati ; Larey, Ariel ; Aknin, Eliel et al. / A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 3211-3217 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS).

@inproceedings{f6c652278c53421cb03e5022f3161727,

title = "A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics",

abstract = "Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.",

keywords = "Decision support system, deep learning, digital pathology, eosinophilic esophagitis, whole slide image segmentation and classification",

author = "Nati Daniel and Ariel Larey and Eliel Aknin and Osswald, {Garrett A.} and Caldwell, {Julie M.} and Mark Rochman and Collins, {Margaret H.} and Yang, {Guang Yu} and Arva, {Nicoleta C.} and Capocelli, {Kelley E.} and Rothenberg, {Marc E.} and Yonatan Savir",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022 ; Conference date: 11-07-2022 Through 15-07-2022",

year = "2022",

doi = "10.1109/EMBC48229.2022.9871086",

language = "English (US)",

series = "Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3211--3217",

booktitle = "44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022",

address = "United States",

}

Daniel, N, Larey, A, Aknin, E, Osswald, GA, Caldwell, JM, Rochman, M, Collins, MH, Yang, GY, Arva, NC, Capocelli, KE, Rothenberg, ME & Savir, Y 2022, A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. in 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, vol. 2022-July, Institute of Electrical and Electronics Engineers Inc., pp. 3211-3217, 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022, Glasgow, United Kingdom, 7/11/22. https://doi.org/10.1109/EMBC48229.2022.9871086

A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. / Daniel, Nati; Larey, Ariel; Aknin, Eliel et al.
44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 3211-3217 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; Vol. 2022-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics

AU - Daniel, Nati

AU - Larey, Ariel

AU - Aknin, Eliel

AU - Osswald, Garrett A.

AU - Caldwell, Julie M.

AU - Rochman, Mark

AU - Collins, Margaret H.

AU - Yang, Guang Yu

AU - Arva, Nicoleta C.

AU - Capocelli, Kelley E.

AU - Rothenberg, Marc E.

AU - Savir, Yonatan

PY - 2022

Y1 - 2022

N2 - Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.

AB - Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.

KW - Decision support system

KW - deep learning

KW - digital pathology

KW - eosinophilic esophagitis

KW - whole slide image segmentation and classification

UR - http://www.scopus.com/inward/record.url?scp=85131554389&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85131554389&partnerID=8YFLogxK

U2 - 10.1109/EMBC48229.2022.9871086

DO - 10.1109/EMBC48229.2022.9871086

M3 - Conference contribution

C2 - 36085661

AN - SCOPUS:85131554389

T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

SP - 3211

EP - 3217

BT - 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022

Y2 - 11 July 2022 through 15 July 2022

ER -

Daniel N, Larey A, Aknin E, Osswald GA, Caldwell JM, Rochman M et al. A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. In 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 3211-3217. (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). doi: 10.1109/EMBC48229.2022.9871086

A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics

Abstract

Publication series

Conference

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this