A de-identifier for medical discharge summaries

Özlem Uzuner*, Tawanda C. Sibanda, Yuan Luo, Peter Szolovits

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

53 Scopus citations

Abstract

Objective: Clinical records contain significant medical information that can be useful to researchers in various disciplines. However, these records also contain personal health information (PHI) whose presence limits the use of the records outside of hospitals. The goal of de-identification is to remove all PHI from clinical records. This is a challenging task because many records contain foreign and misspelled PHI; they also contain PHI that are ambiguous with non-PHI. These complications are compounded by the linguistic characteristics of clinical records. For example, medical discharge summaries, which are studied in this paper, are characterized by fragmented, incomplete utterances and domain-specific language; they cannot be fully processed by tools designed for lay language. Methods and results: In this paper, we show that we can de-identify medical discharge summaries using a de-identifier, Stat De-id, based on support vector machines and local context (F-measure = 97% on PHI). Our representation of local context aids de-identification even when PHI include out-of-vocabulary words and even when PHI are ambiguous with non-PHI within the same corpus. Comparison of Stat De-id with a rule-based approach shows that local context contributes more to de-identification than dictionaries combined with hand-tailored heuristics (F-measure = 85%). Comparison with two well-known named entity recognition (NER) systems, SNoW (F-measure = 94%) and IdentiFinder (F-measure = 36%), on five representative corpora show that when the language of documents is fragmented, a system with a relatively thorough representation of local context can be a more effective de-identifier than systems that combine (relatively simpler) local context with global context. Comparison with a Conditional Random Field De-identifier (CRFD), which utilizes global context in addition to the local context of Stat De-id, confirms this finding (F-measure = 88%) and establishes that strengthening the representation of local context may be more beneficial for de-identification than complementing local with global context.

Original languageEnglish
Pages (from-to)13-35
Number of pages23
JournalArtificial Intelligence in Medicine
Volume42
Issue number1
DOIs
StatePublished - Jan 2008

Keywords

  • Automatic de-identification of narrative patient records
  • Dictionaries
  • Local lexical context
  • Local syntactic context
  • Sentential global context
  • Syntactic information for de-identification

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'A de-identifier for medical discharge summaries'. Together they form a unique fingerprint.

Cite this