A0321
Title: Bias-variance trade-off in feature selection for generalized additive models under concurvity
Authors: Laszlo Kovacs - Corvinus University of Budapest (Hungary) [presenting]
Tibor Keresztely - Corvinus University of Budapest (Hungary)
Zoltan Madari - Corvinus University of Budapest (Hungary)
Abstract: The bias-variance trade-off in statistical learning is about finding a balance between two types of errors: Bias, which is the difference between the expected value of an estimate and the population parameter, and variance, which is the variability of estimates around their expected value. In feature selection, this trade-off plays a critical role in model performance. Selecting too few features may lead to high bias, as the model becomes overly simplistic and may fail to capture important patterns. Conversely, selecting too many features can result in high variance, as the model may overfit to noise in the training set. In generalized additive models (GAMs), concurvity - a non-linear extension of multicollinearity - causes further inflation in the variance of estimators. Some feature selection algorithms attempt to address concurvity, but only for the pairwise cases, so they do not consider when a feature is a multivariate function of several other variables. GAM feature selection algorithms are compared in Monte Carlo simulations under different high-dimensional concurvity scenario groups: No concurvity, pairwise concurvity and multivariate concurvity. The object of the comparisons is the bias-variance trade-off in the spline estimates of GAMs proposed by the examined feature selection algorithms.
