Language models as representations for weakly-supervised NLP tasks

Fei Huang*, Alexander Yates, Arun Ahuja, Douglas C Downey

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Finding the right representation for words is critical for building accurate NLP systems when domain-specific labeled data for the task is scarce. This paper investigates language model representations, in which language models trained on unlabeled corpora are used to generate real-valued feature vectors for words. We investigate ngram models and probabilistic graphical models, including a novel lattice-structured Markov Random Field. Experiments indicate that language model representations outperform traditional representations, and that graphical model representations outperform ngram models, especially on sparse and polysemous words.

Original languageEnglish (US)
Title of host publicationCoNLL 2011 - Fifteenth Conference on Computational Natural Language Learning, Proceedings of the Conference
Pages125-134
Number of pages10
StatePublished - Dec 1 2011
Event15th Conference on Computational Natural Language Learning, CoNLL 2011 - Portland, OR, United States
Duration: Jun 23 2011Jun 24 2011

Publication series

NameCoNLL 2011 - Fifteenth Conference on Computational Natural Language Learning, Proceedings of the Conference

Other

Other15th Conference on Computational Natural Language Learning, CoNLL 2011
CountryUnited States
CityPortland, OR
Period6/23/116/24/11

ASJC Scopus subject areas

  • Artificial Intelligence
  • Linguistics and Language
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Language models as representations for weakly-supervised NLP tasks'. Together they form a unique fingerprint.

  • Cite this

    Huang, F., Yates, A., Ahuja, A., & Downey, D. C. (2011). Language models as representations for weakly-supervised NLP tasks. In CoNLL 2011 - Fifteenth Conference on Computational Natural Language Learning, Proceedings of the Conference (pp. 125-134). (CoNLL 2011 - Fifteenth Conference on Computational Natural Language Learning, Proceedings of the Conference).