Analysis of a probabilistic model of redundancy in unsupervised information extraction

Douglas C Downey*, Oren Etzioni, Stephen Soderland

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Unsupervised Information Extraction (UIE) is the task of extracting knowledge from text without the use of hand-labeled training examples. Because UIE systems do not require human intervention, they can recursively discover new relations, attributes, and instances in a scalable manner. When applied to massive corpora such as the Web, UIE systems present an approach to a primary challenge in artificial intelligence: The automatic accumulation of massive bodies of knowledge. A fundamental problem for a UIE system is assessing the probability that its extracted information is correct. In massive corpora such as the Web, the same extraction is found repeatedly in different documents. How does this redundancy impact the probability of correctness? We present a combinatorial "balls-and-urns" model, called Urns, that computes the impact of sample size, redundancy, and corroboration from multiple distinct extraction rules on the probability that an extraction is correct. We describe methods for estimating Urns's parameters in practice and demonstrate experimentally that for UIE the model's log likelihoods are 15 times better, on average, than those obtained by methods used in previous work. We illustrate the generality of the redundancy model by detailing multiple applications beyond UIE in which Urns has been effective. We also provide a theoretical foundation for Urns's performance, including a theorem showing that PAC Learnability in Urns is guaranteed without hand-labeled data, under certain assumptions.

Original languageEnglish (US)
Pages (from-to)726-748
Number of pages23
JournalArtificial Intelligence
Issue number11
StatePublished - Jul 2010


  • Information extraction
  • Unsupervised
  • World Wide Web

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language
  • Artificial Intelligence


Dive into the research topics of 'Analysis of a probabilistic model of redundancy in unsupervised information extraction'. Together they form a unique fingerprint.

Cite this