Improving identifiability in model calibration using multiple responses

The use of complex computer simulations to design, improve, optimize, or simply to better understand complex systems in many fields of science and engineering is now ubiquitous. However, simulation models are never a perfect representation of physical reality. Two general sources of uncertainty that account for the differences between simulations and experiments are parameter uncertainty and model uncertainty. The former derives from unknown model parameters, while the latter is caused by underlying missing physics, numerical approximations, and other inaccuracies of the computer simulation that exist even if all of the parameters are known. To obtain knowledge of these two sources of uncertainty, data from computer simulations (usually abundant) and data from physical experiments (typically more limited) are often combined using statistical methods. Statistical adjustment of the computer simulation model to account for the two sources of uncertainty is referred to as calibration. We argue that calibration as it is typically implemented, using only a single response variable, is challenging in that it is often extremely difficult to distinguish between the effects of parameter and model uncertainty. However, many different responses (distinct responses and/or the same response measured at different spatial and temporal locations) are automatically calculated in simulations. As multiple responses generally share a mutual dependence on the unknown parameters, they provide valuable information that can improve identifiability of parameter and model uncertainty in calibration, if they are also measured experimentally. In this paper, we explore the use of multiple responses for calibration.