Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation

Wenxin Jiang*, Martin A. Tanner

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

61 Scopus citations

Abstract

We consider hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form ψ(α + xTβ) are mixed. Here ψ(·) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form ψ(h(x)) where h(·) ∈ W2;K0 (a Sobolev class over [0, 1]s). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m-2/s) in Hellinger distance and at a rate of O(m14/s) in Kullback-Leibler divergence, where m is the number of experts, and s is the dimension of the predictor x. We also provide conditions under which the mean-square error of the estimated mean response obtained from the maximum likelihood method converges to zero, as the sample size and the number of experts both increase.

Original languageEnglish (US)
Pages (from-to)987-1011
Number of pages25
JournalAnnals of Statistics
Volume27
Issue number3
DOIs
StatePublished - Jun 1999

Keywords

  • Approximation rate
  • Exponential family
  • Generalized linear models
  • Hellinger distance
  • Hierarchical mixtures-of-experts
  • Kullback-leibler divergence
  • Maximum likelihood estimation
  • Mean square error

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation'. Together they form a unique fingerprint.

Cite this