A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors

Yichi Zhang, Siyu Tao, Wei Chen, Daniel Apley*

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to underlying numerical latent variables (LVs), with the mapped values estimated similarly to the other correlation parameters, and then uses any standard GP covariance function for numerical variables. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as affecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors. Supplementary materials for the article are available online.

Original languageEnglish (US)
JournalTechnometrics
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Latent Variables
Process Modeling
Parameterization
Covariance matrix
Gaussian Process
Physics
Computer simulation
Sufficient Dimension Reduction
Covariance Function
Response Surface
Justification
Simulation Model
Computer Simulation
Imply

Keywords

  • Categorical variables
  • Computer experiments
  • Metamodeling
  • Response surface modeling
  • Surrogate modeling

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Applied Mathematics

Cite this

@article{245c37c65895427eb1fd17fc21b0e32c,
title = "A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors",
abstract = "Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to underlying numerical latent variables (LVs), with the mapped values estimated similarly to the other correlation parameters, and then uses any standard GP covariance function for numerical variables. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as affecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors. Supplementary materials for the article are available online.",
keywords = "Categorical variables, Computer experiments, Metamodeling, Response surface modeling, Surrogate modeling",
author = "Yichi Zhang and Siyu Tao and Wei Chen and Daniel Apley",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/00401706.2019.1638834",
language = "English (US)",
journal = "Technometrics",
issn = "0040-1706",
publisher = "American Statistical Association",

}

A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors. / Zhang, Yichi; Tao, Siyu; Chen, Wei; Apley, Daniel.

In: Technometrics, 01.01.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors

AU - Zhang, Yichi

AU - Tao, Siyu

AU - Chen, Wei

AU - Apley, Daniel

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to underlying numerical latent variables (LVs), with the mapped values estimated similarly to the other correlation parameters, and then uses any standard GP covariance function for numerical variables. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as affecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors. Supplementary materials for the article are available online.

AB - Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to underlying numerical latent variables (LVs), with the mapped values estimated similarly to the other correlation parameters, and then uses any standard GP covariance function for numerical variables. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as affecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors. Supplementary materials for the article are available online.

KW - Categorical variables

KW - Computer experiments

KW - Metamodeling

KW - Response surface modeling

KW - Surrogate modeling

UR - http://www.scopus.com/inward/record.url?scp=85070975451&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070975451&partnerID=8YFLogxK

U2 - 10.1080/00401706.2019.1638834

DO - 10.1080/00401706.2019.1638834

M3 - Article

JO - Technometrics

JF - Technometrics

SN - 0040-1706

ER -