Comparisons with a standard in simulation experiments

Barry L. Nelson*, David Goldsman

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

58 Scopus citations


We Consider the problem of comparing a finite number of stochastic systems with respect to a single system (designated as the "standard") via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any system has larger expected performance than the standard, and if so to identify the best of the alternatives. In this paper we provide two-stage experiment design and analysis procedures to solve the problem for a variety of scenarios, including those in which we encounter unequal variances across systems, as well as those in which we use the variance reduction technique of common random numbers and it is appropriate to do so. The emphasis is added because in some cases common random numbers can be counterproductive when performing comparisons with a standard. We also provide methods for estimating the critical constants required by our procedures, present a portion of an extensive empirical study, and demonstrate one of the procedures via a numerical example.

Original languageEnglish (US)
Pages (from-to)449-463
Number of pages15
JournalManagement Science
Issue number3
StatePublished - Mar 2001


  • Multiple Comparisons
  • Output Analysis
  • Ranking and Selection
  • Simulation

ASJC Scopus subject areas

  • Strategy and Management
  • Management Science and Operations Research


Dive into the research topics of 'Comparisons with a standard in simulation experiments'. Together they form a unique fingerprint.

Cite this