Modeling situated conversational agents as partially observable markov decision processes

A Situated Conversational Agent (SCA) is an agent that engages in dialog about the context within which it is embedded. An SCA is distinguished from non-situated conversational agents by an intimate connection of the agent's dialog to its embedding context, and by intricate dependencies between its linguistic and physical actions. Constructing an SCA that can interact naturally with users while engaged in collaborative physical tasks requires the agent to interleave decision making under uncertainty, action execution, and observation while maximizing expected utility over a sequence of interactions. These requirements can be fulfilled by modeling an SCA as a partially observable Markov decision process (POMDP). We show how POMDPs can be used to formalize and implement psycholinguistic proposals on how situated dialog participants collaborate in order to make and ground dialog contributions.