Enabling Environment Design via Active Indirect Elicitation

Haoqi Zhang*, David C. Parkes

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Many situations arise in which an interested party wishes to affect the decisions of an agent; e.g., a teacher that seeks to promote particular study habits, a Web 2.0 site that seeks to encourage users to contribute content, or an online retailer that seeks to encourage consumers to write reviews. In the problem of environment design, one assumes an interested party who is able to alter limited aspects of the environment for the purpose of promoting desirable behaviors. A critical aspect of environment design is understanding preferences, but by assumption direct queries are unavailable. We work in the inverse reinforcement learning framework, adopting here the idea of active indirect preference elicitation to learn the reward function of the agent by observing behavior in response to incentives. We show that the process is convergent and obtain desirable bounds on the number of elicitation rounds. We briefly discuss generalizations of the elicitation method to other forms of environment design, e.g., modifying the state space, transition model, and available actions.

Original languageEnglish (US)
Title of host publicationMultidisciplinary Workshop on Advances in Preference Handling - Papers from the 2008 AAAI Workshop, Technical Report
Number of pages6
StatePublished - Dec 1 2008
Event2008 AAAI Workshop - Chicago, IL, United States
Duration: Jul 13 2008Jul 13 2008


Other2008 AAAI Workshop
Country/TerritoryUnited States
CityChicago, IL

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'Enabling Environment Design via Active Indirect Elicitation'. Together they form a unique fingerprint.

Cite this