A1028
Title: Data adaptive random forest kernels via dimension reduction
Authors: Tomoshige Nakamura - Juntendo University (Japan) [presenting]
Hiroshi Shiraishi - Keio University (Japan)
Abstract: A past study proposed random forests as an ensemble method that uses regression trees/decision trees as weak learners, which has demonstrated high accuracy in various regression and classification problems across different fields. Another study provided a new interpretation of random forests as a method for estimating data-adaptive kernel weighting functions based on a loss function. It showed that functional parameters characterized by local estimating equations can be estimated using random forests. They also proved the asymptotic normality for those estimators. Building upon these results, the asymptotic behavior of the kernel weighting function generated by random forests is investigated. It is found that random forests kernels converge to the Laplace kernel when the features are one-dimensional, while in the multidimensional case, they become the product of Laplace kernels for each feature dimension. This fact suggests that the expressive power of data-adaptive kernels generated by random forests is limited, and there is a problem of significantly lower accuracy when estimating functionals that can be represented by the sum of features, for example. The problem is demonstrated to be resolved by modifying the splitting rule of the trees constituting the random forest from a threshold-based split on a single variable to a data-adaptive split obtained through sufficient dimension reduction.
