A1617
Title: Predictive modelling with ensembles of projected nearest neighbors
Authors: David Hofmeyr - Lancaster University (United Kingdom) [presenting]
Abstract: Nearest neighbors (NN) based estimators are one of the most popular among non-parametric smoothing techniques, largely for their simplicity and (relative) computational efficiency. However, their successful use within ensemble models has been overshadowed by the far more popular decision tree (DT) based models; such as random forests (RF) and gradient boosting (GB) models, which use DTs as base learners. Where DTs are advantaged over (most) other non-parametric smoothers is in how they adaptively determine smoothing neighbourhoods, which are optimised with reference to the problem objective. This adaptiveness naturally increases the complexity of the resulting model, increasing its variance, which is why their greatest successes have been in the context of the ensemble models mentioned previously. Inspired by this, the utility of using projection methods are investigated, which are problem-specific, to allow the neighborhood search in NNs to similarly be adaptive to the problem objective. The resulting models are simple to implement and computationally efficient, as well as being versatile and widely applicable. In addition, their accuracy in prediction tasks, such as classification and regression, is competitive with the DT alternatives.
