B0620
Title: Combining a smooth information criterion with neural networks
Authors: Andrew McInerney - University of Limerick (Ireland) [presenting]
Kevin Burke - University of Limerick (Ireland)
David Ruegamer - LMU Munich (Germany)
Abstract: Feedforward neural networks (FNNs) can be viewed through a statistical lens as parametric non-linear regression models, where the covariates are mapped to the response through a series of weighted summations and non-linear functions. They are a highly flexible class of models and have been very successful in the prediction of complex problems. However, interpretation of these models can be difficult, primarily due to their relatively large number of parameters. One approach to aid their interpretability is to ensure sparse solutions. The use of the smooth information criterion (SIC) is proposed, which uses a smooth approximation to the $L0$ norm embedded within an information-criterion-based penalised likelihood, to sparsify the FNN model. This approach is computationally advantageous as the penalty parameter is known from the outset, e.g., it is $log(n)$ for the BIC, and, hence, avoids the challenge of tuning. Furthermore, the SIC is extended to group penalisation to enforce structured sparsity, allowing for automatic variable selection among the input nodes and the determination of model complexity through the hidden nodes. The favourable performance of the method is shown in simulation studies and an application to real data is investigated.
