Lyapunov Conditions for Differentiability of Markov Chain Expectations

Chang Han Rhee*, Peter W. Glynn

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

We consider a family of Markov chains whose transition dynamics are affected by model parameters. Understanding the parametric dependence of (complex) performance measures of such Markov chains is often of significant interest. The derivatives and their continuity of the performance measures w.r.t. the parameters play important roles, for example, in numerical optimization of the performance measures, and quantification of the uncertainties in the performance measures when there are uncertainties in the parameters from the statistical estimation procedures. In this paper, we establish conditions that guarantee the smoothness of various types of intractable performance measures—such as the stationary and random horizon discounted performance measures—of general state space Markov chains and provide probabilistic representations for the derivatives.

Original languageEnglish (US)
Pages (from-to)2019-2042
Number of pages24
JournalMathematics of Operations Research
Volume48
Issue number4
DOIs
StatePublished - 2023

Keywords

  • Markov chain
  • derivative estimation
  • sensitivity analysis

ASJC Scopus subject areas

  • General Mathematics
  • Computer Science Applications
  • Management Science and Operations Research

Fingerprint

Dive into the research topics of 'Lyapunov Conditions for Differentiability of Markov Chain Expectations'. Together they form a unique fingerprint.

Cite this