TY - JOUR
T1 - Combinatorial design of a stochastic Markov decision process
AU - Dimitrov, Nedialko B.
AU - Morton, David P.
N1 - Funding Information:
We thank Alexander Moffett and Sahotra Sarkar from the University of Texas Section for Integrative Biology for their help in constructing the malaria application. This research was supported by the National Science Foundation under Grants CBET-0736231 and CMMI-0653916.
PY - 2009
Y1 - 2009
N2 - We consider a problem in which we seek to optimally design a Markov decision process (MDP). That is, subject to resource constraints we first design the action sets that will be available in each state when we later optimally control the process. The control policy is subject to additional constraints governing state-action pair frequencies, and we allow randomized policies. When the design decision is made, we are uncertain of some of the parameters governing the MDP, but we assume a distribution for these stochastic parameters is known. We focus on transient MDPs with a finite number of states and actions. We formulate, analyze and solve a two-stage stochastic integer program that yields an optimal design. A simple example threads its way through the paper to illustrate the development. The paper concludes with a larger application involving optimal design of malaria intervention strategies in Nigeria.
AB - We consider a problem in which we seek to optimally design a Markov decision process (MDP). That is, subject to resource constraints we first design the action sets that will be available in each state when we later optimally control the process. The control policy is subject to additional constraints governing state-action pair frequencies, and we allow randomized policies. When the design decision is made, we are uncertain of some of the parameters governing the MDP, but we assume a distribution for these stochastic parameters is known. We focus on transient MDPs with a finite number of states and actions. We formulate, analyze and solve a two-stage stochastic integer program that yields an optimal design. A simple example threads its way through the paper to illustrate the development. The paper concludes with a larger application involving optimal design of malaria intervention strategies in Nigeria.
KW - Action space design
KW - Markov decision process
KW - Stochastic optimization
UR - http://www.scopus.com/inward/record.url?scp=77955453073&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77955453073&partnerID=8YFLogxK
U2 - 10.1007/978-0-387-88843-9_9
DO - 10.1007/978-0-387-88843-9_9
M3 - Article
AN - SCOPUS:77955453073
VL - 47
SP - 167
EP - 193
JO - Operations Research/ Computer Science Interfaces Series
JF - Operations Research/ Computer Science Interfaces Series
SN - 1387-666X
ER -