Penalized likelihood estimation for semiparametric mixed models, withapplication to alcohol treatment research

Jinsong Chen, Lei Liu*, Bankole A. Johnson, John O'Quigley

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


In this article, we implement a practical computational method for various semiparametric mixed effects models, estimating nonlinear functions by penalized splines. We approximate the integration of the penalized likelihood with respect to random effects with the use of adaptive Gaussian quadrature, which we can conveniently implement in SAS procedure NLMIXED. We carry out the selection of smoothing parameters through approximated generalized cross-validation scores. Our method has two advantages: (1) the estimation is more accurate than the current available quasi-likelihood method for sparse data, for example, binary data; and (2) it can be used in fitting more sophisticated models. We show the performance of our approach in simulation studies with longitudinal outcomes from three settings: binary, normal data after Box-Cox transformation, and count data with log-Gamma random effects. We also develop an estimation method for a longitudinal two-part nonparametric random effects model and apply it to analyze repeated measures of semicontinuous daily drinking records in a randomized controlled trial of topiramate.

Original languageEnglish (US)
Pages (from-to)335-346
Number of pages12
JournalStatistics in Medicine
Issue number2
StatePublished - Jan 30 2013


  • Generalized linear mixed models (GLMMs)
  • Laplace approximation
  • Logistic models
  • Longitudinal data analysis
  • Non-normal random effects

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability


Dive into the research topics of 'Penalized likelihood estimation for semiparametric mixed models, withapplication to alcohol treatment research'. Together they form a unique fingerprint.

Cite this