## Abstract

We consider hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form ψ(α + x^{T}β) are mixed. Here ψ(·) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form ψ(h(x)) where h(·) ∈ W^{∞}_{2;K0} (a Sobolev class over [0, 1]^{s}). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m^{-2/s}) in Hellinger distance and at a rate of O(m^{14/s}) in Kullback-Leibler divergence, where m is the number of experts, and s is the dimension of the predictor x. We also provide conditions under which the mean-square error of the estimated mean response obtained from the maximum likelihood method converges to zero, as the sample size and the number of experts both increase.

Original language | English (US) |
---|---|

Pages (from-to) | 987-1011 |

Number of pages | 25 |

Journal | Annals of Statistics |

Volume | 27 |

Issue number | 3 |

DOIs | |

State | Published - Jun 1999 |

## Keywords

- Approximation rate
- Exponential family
- Generalized linear models
- Hellinger distance
- Hierarchical mixtures-of-experts
- Kullback-leibler divergence
- Maximum likelihood estimation
- Mean square error

## ASJC Scopus subject areas

- Statistics and Probability
- Statistics, Probability and Uncertainty