A0833
Title: Unity forests: Improving interaction modelling and interpretability in random forests
Authors: Roman Hornung - University of Munich (Germany) [presenting]
Abstract: Random forests (RFs) are widely used for prediction and variable importance analysis and are often assumed to capture interactions via recursive splitting. However, since the splits are chosen locally at each node, RFs capture interactions only when at least one involved covariate has a notable marginal effect. Unity forests (UFOs) are introduced, an RF variant designed to exploit interactions involving covariates without marginal effects. In UFOs, the first few splits of each tree (by default three) are optimized jointly across a random covariate subset to form a "tree root" that better captures such interactions; the remainder is grown conventionally. A new variable importance measure (VIM) is also proposed - the unity VIM - based on the out-of-bag split criterion values from the tree roots. Only a small fraction of root splits with the highest in-bag values are considered per covariate, reflecting that covariates involved in interactions tend to be influential in relatively few trees. In a simulation study, the unity VIM consistently ranked interacting covariates without marginal effects above non-influential ones - unlike conventional RF-based VIMs. In a large-scale real data-based comparison, UFOs improved predictive accuracy and discrimination over standard RFs, with similar calibration. Finally, selecting representative trees is explored per covariate from the tree roots, offering interpretable insight into each covariate's individual or interactive predictive role.
