A0361
Title: Scalable robust estimators for non-parametric regression models
Authors: Matias Salibian-Barrera - The University of British Columbia (Canada) [presenting]
Xiaomeng Ju - The University of British Columbia (Canada)
Abstract: Many robust estimators for nonparametric regression models have been proposed in the literature. Unfortunately, most non-parametric regression estimators generally do not scale well when the number of explanatory variables is relatively large (c.f. the curse of dimensionality). Additive models can avoid this problem, at the expense of having to impose a strong structure in the regression function. A different family of non-parametric regression estimators is given by gradient boosting, which constructs a regression predictor using a linear combination of simple base learners (e.g. regression trees), which can be used effectively even with many covariates. A robust variant of gradient boosting for regression problems can be obtained by minimizing a properly defined loss function. To avoid relying on ad-hoc estimates of the residual scale that change in every iteration, we use a two-stage approach (as with MM-estimators for linear regression): we first minimize a robust residual scale estimator, and then improve it by optimizing an M-type loss function. Simulation studies and several data analyses show that, when no atypical observations are present, the robust boosting approach works as well as the standard gradient boosting one with a squared loss. As expected, when the data contain outliers the robust boosting estimator outperforms existing alternatives.
