Definition modeling

Learning to define word embeddings in natural language

Thanapon Noraset, Chen Liang, Lawrence A Birnbaum, Douglas C Downey

Research output: Contribution to conferencePaper

4 Citations (Scopus)

Abstract

Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a characterlevel convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.

Original languageEnglish (US)
Pages3259-3266
Number of pages8
StatePublished - Jan 1 2017
Event31st AAAI Conference on Artificial Intelligence, AAAI 2017 - San Francisco, United States
Duration: Feb 4 2017Feb 10 2017

Other

Other31st AAAI Conference on Artificial Intelligence, AAAI 2017
CountryUnited States
CitySan Francisco
Period2/4/172/10/17

Fingerprint

Semantics
Recurrent neural networks
Glossaries
Convolution
Error analysis
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Noraset, T., Liang, C., Birnbaum, L. A., & Downey, D. C. (2017). Definition modeling: Learning to define word embeddings in natural language. 3259-3266. Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States.
Noraset, Thanapon ; Liang, Chen ; Birnbaum, Lawrence A ; Downey, Douglas C. / Definition modeling : Learning to define word embeddings in natural language. Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States.8 p.
@conference{ae02cfa9f3514a42b158e92ece4df982,
title = "Definition modeling: Learning to define word embeddings in natural language",
abstract = "Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a characterlevel convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.",
author = "Thanapon Noraset and Chen Liang and Birnbaum, {Lawrence A} and Downey, {Douglas C}",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
pages = "3259--3266",
note = "31st AAAI Conference on Artificial Intelligence, AAAI 2017 ; Conference date: 04-02-2017 Through 10-02-2017",

}

Noraset, T, Liang, C, Birnbaum, LA & Downey, DC 2017, 'Definition modeling: Learning to define word embeddings in natural language' Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States, 2/4/17 - 2/10/17, pp. 3259-3266.

Definition modeling : Learning to define word embeddings in natural language. / Noraset, Thanapon; Liang, Chen; Birnbaum, Lawrence A; Downey, Douglas C.

2017. 3259-3266 Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Definition modeling

T2 - Learning to define word embeddings in natural language

AU - Noraset, Thanapon

AU - Liang, Chen

AU - Birnbaum, Lawrence A

AU - Downey, Douglas C

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a characterlevel convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.

AB - Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a characterlevel convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.

UR - http://www.scopus.com/inward/record.url?scp=85030475663&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030475663&partnerID=8YFLogxK

M3 - Paper

SP - 3259

EP - 3266

ER -

Noraset T, Liang C, Birnbaum LA, Downey DC. Definition modeling: Learning to define word embeddings in natural language. 2017. Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States.