Learning from unannotated QA pairs to analogically disambiguate and answer questions

Maxwell Crouse, Clifton McFate, Kenneth D Forbus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Creating systems that can learn to answer natural language questions has been a longstanding challenge for artificial intelligence. Most prior approaches focused on producing a specialized language system for a particular domain and dataset, and they required training on a large corpus manually annotated with logical forms. This paper introduces an analogy-based approach that instead adapts an existing general purpose semantic parser to answer questions in a novel domain by jointly learning disambiguation heuristics and query construction templates from purely textual question-answer pairs. Our technique uses possible semantic interpretations of the natural language questions and answers to constrain a query-generation procedure, producing cases during training that are subsequently reused via analogical retrieval and composed to answer test questions. Bootstrapping an existing semantic parser in this way significantly reduces the number of training examples needed to accurately answer questions. We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2-folds, and achieves above 50% accuracy with as few as 10 examples.

Original languageEnglish (US)
Title of host publication32nd AAAI Conference on Artificial Intelligence, AAAI 2018
PublisherAAAI press
Pages654-662
Number of pages9
ISBN (Electronic)9781577358008
StatePublished - Jan 1 2018
Event32nd AAAI Conference on Artificial Intelligence, AAAI 2018 - New Orleans, United States
Duration: Feb 2 2018Feb 7 2018

Publication series

Name32nd AAAI Conference on Artificial Intelligence, AAAI 2018

Other

Other32nd AAAI Conference on Artificial Intelligence, AAAI 2018
CountryUnited States
CityNew Orleans
Period2/2/182/7/18

Fingerprint

Semantics
Artificial intelligence

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Crouse, M., McFate, C., & Forbus, K. D. (2018). Learning from unannotated QA pairs to analogically disambiguate and answer questions. In 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 (pp. 654-662). (32nd AAAI Conference on Artificial Intelligence, AAAI 2018). AAAI press.
Crouse, Maxwell ; McFate, Clifton ; Forbus, Kenneth D. / Learning from unannotated QA pairs to analogically disambiguate and answer questions. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press, 2018. pp. 654-662 (32nd AAAI Conference on Artificial Intelligence, AAAI 2018).
@inproceedings{4f6e701a65cb4919b4f71e94d5ee056f,
title = "Learning from unannotated QA pairs to analogically disambiguate and answer questions",
abstract = "Creating systems that can learn to answer natural language questions has been a longstanding challenge for artificial intelligence. Most prior approaches focused on producing a specialized language system for a particular domain and dataset, and they required training on a large corpus manually annotated with logical forms. This paper introduces an analogy-based approach that instead adapts an existing general purpose semantic parser to answer questions in a novel domain by jointly learning disambiguation heuristics and query construction templates from purely textual question-answer pairs. Our technique uses possible semantic interpretations of the natural language questions and answers to constrain a query-generation procedure, producing cases during training that are subsequently reused via analogical retrieval and composed to answer test questions. Bootstrapping an existing semantic parser in this way significantly reduces the number of training examples needed to accurately answer questions. We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2-folds, and achieves above 50{\%} accuracy with as few as 10 examples.",
author = "Maxwell Crouse and Clifton McFate and Forbus, {Kenneth D}",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
series = "32nd AAAI Conference on Artificial Intelligence, AAAI 2018",
publisher = "AAAI press",
pages = "654--662",
booktitle = "32nd AAAI Conference on Artificial Intelligence, AAAI 2018",

}

Crouse, M, McFate, C & Forbus, KD 2018, Learning from unannotated QA pairs to analogically disambiguate and answer questions. in 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, AAAI press, pp. 654-662, 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, New Orleans, United States, 2/2/18.

Learning from unannotated QA pairs to analogically disambiguate and answer questions. / Crouse, Maxwell; McFate, Clifton; Forbus, Kenneth D.

32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press, 2018. p. 654-662 (32nd AAAI Conference on Artificial Intelligence, AAAI 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Learning from unannotated QA pairs to analogically disambiguate and answer questions

AU - Crouse, Maxwell

AU - McFate, Clifton

AU - Forbus, Kenneth D

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Creating systems that can learn to answer natural language questions has been a longstanding challenge for artificial intelligence. Most prior approaches focused on producing a specialized language system for a particular domain and dataset, and they required training on a large corpus manually annotated with logical forms. This paper introduces an analogy-based approach that instead adapts an existing general purpose semantic parser to answer questions in a novel domain by jointly learning disambiguation heuristics and query construction templates from purely textual question-answer pairs. Our technique uses possible semantic interpretations of the natural language questions and answers to constrain a query-generation procedure, producing cases during training that are subsequently reused via analogical retrieval and composed to answer test questions. Bootstrapping an existing semantic parser in this way significantly reduces the number of training examples needed to accurately answer questions. We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2-folds, and achieves above 50% accuracy with as few as 10 examples.

AB - Creating systems that can learn to answer natural language questions has been a longstanding challenge for artificial intelligence. Most prior approaches focused on producing a specialized language system for a particular domain and dataset, and they required training on a large corpus manually annotated with logical forms. This paper introduces an analogy-based approach that instead adapts an existing general purpose semantic parser to answer questions in a novel domain by jointly learning disambiguation heuristics and query construction templates from purely textual question-answer pairs. Our technique uses possible semantic interpretations of the natural language questions and answers to constrain a query-generation procedure, producing cases during training that are subsequently reused via analogical retrieval and composed to answer test questions. Bootstrapping an existing semantic parser in this way significantly reduces the number of training examples needed to accurately answer questions. We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2-folds, and achieves above 50% accuracy with as few as 10 examples.

UR - http://www.scopus.com/inward/record.url?scp=85060496073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060496073&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85060496073

T3 - 32nd AAAI Conference on Artificial Intelligence, AAAI 2018

SP - 654

EP - 662

BT - 32nd AAAI Conference on Artificial Intelligence, AAAI 2018

PB - AAAI press

ER -

Crouse M, McFate C, Forbus KD. Learning from unannotated QA pairs to analogically disambiguate and answer questions. In 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press. 2018. p. 654-662. (32nd AAAI Conference on Artificial Intelligence, AAAI 2018).