Human–machine partnership with artificial intelligence for chest radiograph diagnosis

Bhavik N. Patel*, Louis Rosenberg, Gregg Willcox, David Baltaxe, Mimi Lyons, Jeremy Irvin, Pranav Rajpurkar, Timothy Amrhein, Rajan Gupta, Safwan Halabi, Curtis Langlotz, Edward Lo, Joseph Mammarappallil, A. J. Mariano, Geoffrey Riley, Jayne Seekins, Luyao Shen, Evan Zucker, Matthew Lungren

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

113 Scopus citations


Human-in-the-loop (HITL) AI may enable an ideal symbiosis of human experts and AI models, harnessing the advantages of both while at the same time overcoming their respective limitations. The purpose of this study was to investigate a novel collective intelligence technology designed to amplify the diagnostic accuracy of networked human groups by forming real-time systems modeled on biological swarms. Using small groups of radiologists, the swarm-based technology was applied to the diagnosis of pneumonia on chest radiographs and compared against human experts alone, as well as two state-of-the-art deep learning AI models. Our work demonstrates that both the swarm-based technology and deep-learning technology achieved superior diagnostic accuracy than the human experts alone. Our work further demonstrates that when used in combination, the swarm-based technology and deep-learning technology outperformed either method alone. The superior diagnostic accuracy of the combined HITL AI solution compared to radiologists and AI alone has broad implications for the surging clinical AI deployment and implementation strategies in future practice.

Original languageEnglish (US)
Article number111
Journalnpj Digital Medicine
Issue number1
StatePublished - Dec 1 2019

ASJC Scopus subject areas

  • Health Information Management
  • Health Informatics
  • Medicine (miscellaneous)
  • Computer Science Applications


Dive into the research topics of 'Human–machine partnership with artificial intelligence for chest radiograph diagnosis'. Together they form a unique fingerprint.

Cite this