Using ranking and selection to "clean UP" after simulation optimization

Justin Boesel*, Barry L. Nelson, Seong Hee Kim

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

121 Scopus citations


In this paper we address the problem of finding the simulated system with the best (maximum or minimum) expected performance when the number of systems is large and initial samples from each system have already been taken. This problem may be encountered when a heuristic search procedure - perhaps one originally designed for use in a deterministic environment - has been applied in a simulation-optimization context. Because of stochastic variation, the system with the best sample mean at the end of the search procedure may not coincide with the true best system encountered during the search. This paper develops statistical procedures that return the best system encountered by the search (or one near the best) with a prespecified probability. We approach this problem using combinations of statistical subset selection and indifference-zone ranking procedures. The subset-selection procedures, which use only the data already collected, screen out the obviously inferior systems, while the indifference-zone procedures, which require additional simulation effort, distinguish the best from the less obviously inferior systems.

Original languageEnglish (US)
Pages (from-to)814-825
Number of pages12
JournalOperations Research
Issue number5
StatePublished - Sep 2003


  • Programming/stochastic: terminal inference
  • Simulation, efficiency: large-scale screening
  • Simulation, statistical analysis: selecting the best system

ASJC Scopus subject areas

  • Computer Science Applications
  • Management Science and Operations Research


Dive into the research topics of 'Using ranking and selection to "clean UP" after simulation optimization'. Together they form a unique fingerprint.

Cite this