COMPSTAT 2024: Start Registration
View Submission - COMPSTAT2024
A0451
Title: A penalized maximum likelihood estimation for hidden Markov models to address latent state separation Authors:  Luca Brusa - University of Milano-Bicocca (Italy) [presenting]
Francesco Bartolucci - University of Perugia (Italy)
Fulvia Pennoni - University of Milano-Bicocca (Italy)
Romina Peruilh Bagolini - University of Perugia (Italy)
Abstract: In analyzing longitudinal data, the focus is usually on the evolution of a characteristic of interest over time, which is measured by occasion-specific response variables. To analyze such data, the hidden Markov model assumes a latent process typically following a Markov chain of the first order and affecting the distribution of the response variables. It may include both time-constant and time-varying covariates, which can affect the conditional distribution of the response variable (measurement model). The latent process accounts for unobserved heterogeneity when covariates cannot fully explain the variability among responses. It also allows the consideration of state dependence by including the lagged responses among the covariates. When these covariates do not fully explain the heterogeneity between individuals, the parameters corresponding to the effect of the latent states may be very large, leading to widely separated states that cause instability of the estimated parameters. A penalized likelihood approach is implemented by modifying the M-step of the expectation-maximization algorithm. In addition, a cross-validation method is proposed to jointly select the number of latent states and the strength of the penalty term. The asymptotic properties of the estimator are examined via simulation, and the proposal is illustrated to estimate the effect of covariates on hypotension occurrence during anesthesia performed before surgery.