Markov decision processes with exogenous variables

Robert L. Bray*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


I present two algorithms for solving dynamic programs with exogenous variables: endogenous value iteration and endogenous policy iteration. These algorithms are always at least as fast as relative value iteration and relative policy iteration, and they are faster when the endogenous variables converge to their stationary distributions sooner than the exogenous variables.

Original languageEnglish (US)
Pages (from-to)4598-4606
Number of pages9
JournalManagement Science
Issue number10
StatePublished - Oct 1 2019


  • Dynamic programming
  • Endogenous value iteration
  • Exogenous variables
  • Markov decision process
  • Relative value iteration

ASJC Scopus subject areas

  • Strategy and Management
  • Management Science and Operations Research

Fingerprint Dive into the research topics of 'Markov decision processes with exogenous variables'. Together they form a unique fingerprint.

Cite this