Enhancing recall in automated record screening: A resampling algorithm

Zhipeng Hou*, Elizabeth Tipton

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review


Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with high priority. In previous studies, screening prioritization is often referred to as automatic literature screening or automatic literature identification. Numerous screening prioritization methods have been proposed in recent years. However, there is a lack of screening prioritization methods with reliable performance. Our objective is to develop a screening prioritization algorithm with reliable performance for practical use, for example, an algorithm that guarantees an 80% chance of identifying at least (Formula presented.) of the relevant records. Based on a target-based method proposed in Cormack and Grossman, we propose a screening prioritization algorithm using sampling with replacement. The algorithm is a wrapper algorithm that can work with any current screening prioritization algorithm to guarantee the performance. We prove, with mathematics and probability theory, that the algorithm guarantees the performance. We also run numeric experiments to test the performance of our algorithm when applied in practice. The numeric experiment results show this algorithm achieve reliable performance under different circumstances. The proposed screening prioritization algorithm can be reliably used in real world research synthesis tasks.

Original languageEnglish (US)
Pages (from-to)372-383
Number of pages12
JournalResearch synthesis methods
Issue number3
StatePublished - May 2024


  • automatic screening algorithm
  • data mining
  • literature screen
  • machine learning
  • text mining

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'Enhancing recall in automated record screening: A resampling algorithm'. Together they form a unique fingerprint.

Cite this