Enabling Environment Design via Active Indirect Elicitation

Haoqi Zhang*, David C. Parkes

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many situations arise in which an interested party wishes to affect the decisions of an agent; e.g., a teacher that seeks to promote particular study habits, a Web 2.0 site that seeks to encourage users to contribute content, or an online retailer that seeks to encourage consumers to write reviews. In the problem of environment design, one assumes an interested party who is able to alter limited aspects of the environment for the purpose of promoting desirable behaviors. A critical aspect of environment design is understanding preferences, but by assumption direct queries are unavailable. We work in the inverse reinforcement learning framework, adopting here the idea of active indirect preference elicitation to learn the reward function of the agent by observing behavior in response to incentives. We show that the process is convergent and obtain desirable bounds on the number of elicitation rounds. We briefly discuss generalizations of the elicitation method to other forms of environment design, e.g., modifying the state space, transition model, and available actions.

Original languageEnglish (US)
Title of host publicationMultidisciplinary Workshop on Advances in Preference Handling - Papers from the 2008 AAAI Workshop, Technical Report
Pages140-145
Number of pages6
VolumeWS-08-09
StatePublished - Dec 1 2008
Event2008 AAAI Workshop - Chicago, IL, United States
Duration: Jul 13 2008Jul 13 2008

Other

Other2008 AAAI Workshop
Country/TerritoryUnited States
CityChicago, IL
Period7/13/087/13/08

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Enabling Environment Design via Active Indirect Elicitation'. Together they form a unique fingerprint.

Cite this