A computer vision pipeline for automated determination of cardiac structure and function and detection of disease by two-dimensional echocardiography

Jeffrey Zhang, Sravani Gajjala, Pulkit Agrawal, Geoffrey H. Tison, Laura A. Hallock, Lauren Beussink-Nelson, Mats H. Lassen, Eugene Fan, Mandar A. Aras, Cha Randle Jordan, Kirsten E. Fleischmann, Michelle Melisko, Atif Qasim, Alexei Efros, Sanjiv J. Shah, Ruzena Bajcsy, Rahul C. Deo*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function by non-experts in the primary care and rural setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation. Our approach entailed: 1) preprocessing of complete echo studies; 2) convolutional neural networks (CNN) for view identification, image segmentation, and phasing of the cardiac cycle; 3) quantification of chamber volumes and left ventricular mass; 4) particle tracking to compute longitudinal strain; and 5) targeted disease detection. CNNs accurately identified views (e.g. 99% for apical 4-chamber) and segmented individual cardiac chambers. The resulting cardiac structure measurements agreed with study report values [e.g. median absolute deviations (MAD) of 11.8 g/kg/m2 for left ventricular mass index and 7.7 mL/kg/m2 for left ventricular diastolic volume index, derived from 1319 and 2918 studies, respectively]. In terms of cardiac function, we computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values [for ejection fraction, MAD=5.3%, N=3101 studies; for strain, MAD=1.5% (n=197) and 1.6% (n=110)], and demonstrated applicability to serial monitoring of breast cancer patients for trastuzumab cardiotoxicity. Overall, we found that, compared to manual measurements, automated measurements had superior performance across seven internal consistency metrics (e.g. the correlation of left ventricular diastolic volumes with left atrial volumes) with an average increase in the absolute Spearman correlation coefficient of 0.05 (p=0.02). Finally, we used CNNs to develop disease detection algorithms for hypertrophic cardiomyopathy and cardiac amyloidosis, with C-statistics of 0.93 and 0.84, respectively. Our pipeline lays the groundwork for using automated interpretation to support point-of-care handheld cardiac ultrasound and large-scale analysis of the millions of echos archived within healthcare systems.

Original languageEnglish (US)
JournalUnknown Journal
StatePublished - Jun 22 2017

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'A computer vision pipeline for automated determination of cardiac structure and function and detection of disease by two-dimensional echocardiography'. Together they form a unique fingerprint.

Cite this