Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation

We consider hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form ψ(α + x^Tβ) are mixed. Here ψ(·) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form ψ(h(x)) where h(·) ∈ W^∞_2;K0 (a Sobolev class over [0, 1]^s). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m^-2/s) in Hellinger distance and at a rate of O(m^14/s) in Kullback-Leibler divergence, where m is the number of experts, and s is the dimension of the predictor x. We also provide conditions under which the mean-square error of the estimated mean response obtained from the maximum likelihood method converges to zero, as the sample size and the number of experts both increase.