A0307
Title: Revisiting parameter estimation and model selection in Gaussian mixtures of experts
Authors: Trung Tin Nguyen - Queensland University of Technology (Australia) [presenting]
Abstract: Parameter estimation and model selection in mixtures of experts remain long-standing challenges in machine learning and statistics. These challenges arise primarily from the inclusion of covariates in gating functions and expert networks, leading to intrinsic interactions governed by partial differential equations with respect to model parameters. To address these issues, novel Voronoi loss functions are introduced that effectively capture the heterogeneity in parameter estimation rates. Additionally, a general framework is proposed for model selection in Gaussian mixture of experts models, either by leveraging Bayesian nonparametric methods to eliminate the need to predefine the number of experts or by employing a dendrogram-based approach on mixing measures to mitigate sensitivity to hyperparameter selection. Experimental results on simulated data validate the effectiveness of our approach, demonstrating its ability to empirically support theoretical findings, accurately estimate the regression function, and recover the true number of experts.
