Simultaneous fixed and random effects selection in finite mixture of linear mixed-effects models

In this article, we study finite mixtures of linear mixed-effects (FMLME) models that are useful for longitudinal regression modelling in the presence of heterogeneity in both fixed and random effects. These models are computationally challenging when the number of covariates is large, and traditional variable selection techniques become expensive to implement. We introduce a penalized likelihood approach, and propose a nested EM algorithm for efficient numerical computations. The resulting estimators are shown to possess consistency and sparsity properties, and to be asymptotically normally distributed. We illustrate the performance of our method through simulations and a real data example.