EcoSta 2021: Start Registration
View Submission - EcoSta2021
A0579
Title: Statistical learning via spectrally sparse smoothers Authors:  Nathaniel Helwig - University of Minnesota (United States) [presenting]
Abstract: Statistical learning methods seek to find reliable prediction rules that can produce insightful discoveries about complex datasets. Many statistical learning methods use penalized likelihood estimation to estimate such prediction rules, which adds a roughness penalty to the (negative) log-likelihood function. The influence of the roughness penalty on the prediction rule is typically controlled via tuning parameters, which are selected using some form of cross-validation. Popular examples of penalized likelihood methods include regularized linear regression models (e.g., ridge, lasso, elastic net) and regularized nonparametric regression models (e.g., penalized splines, generalized additive models, smoothing spline ANOVA models). To combine the benefits of regression selection and smoothing methods, we propose a spectral parameterization of a penalized spline, which allows for an efficient application of elastic net regression to smooth and select eigenvectors of a kernel matrix. The classic solution for a penalized spline is a special case of the proposed kernel eigenvector smoothing and selection operator (kesso). Extensions for tensor product smoothers are developed for both the GAM and SSANOVA frameworks. Using simulated and real data examples, we demonstrate that the kesso offers practical and computational gains over typical approaches for fitting GAMs, SSANOVA models, and elastic net penalized GLMs.