B1393
Title: Locally linear forests
Authors: Rina Friedberg - Stanford University (United States) [presenting]
Julie Tibshirani - Palantir Technologies (United States)
Susan Athey - Stanford University (United States)
Stefan Wager - Stanford University (United States)
Abstract: Random forests are a powerful and ubiquitous predictive method, increasingly considered as an adaptive approach for non-parametric statistical estimation. A limitation of Breiman's forests, however, is that they cannot take advantage of smoothness properties of the underlying signal. This leads to unstable estimates and noticeably suboptimal prediction in the presence of strong linear effects. Local linear regression is another popular method that performs well in this setting, but that traditionally employs non-adaptive kernels and is moreover subject to an acute curse of dimensionality. Drawing on the strengths of these techniques, we introduce locally linear forests, which use an adaptation of Breiman's algorithm to motivate a data-adaptive kernel that can then be plugged into a locally weighted regularized regression. In our experiments, we find that locally linear forests improve upon traditional regression forests in the presence of strong, smooth effects. We contrast our procedure with ``model forests'', which aggregate within-leaf predictions from many different regression models, and discuss problem-specific splitting rules and inferential methods.
