A0795
Title: Generalization analysis of deep CNNs under maximum correntropy criterion
Authors: Zhiying Fang - Shenzhen Polytechnic University (China) [presenting]
Jun Fan - Hong Kong Baptist University (Hong Kong)
Yingqiao Zhang - Hong Kong Baptist University (Hong Kong)
Abstract: Convolutional neural networks (CNNs) have gained immense popularity in recent years, finding their utility in diverse fields such as image recognition, natural language processing, and bioinformatics. Despite the remarkable progress made in deep learning theory, most studies on CNNs, especially in regression tasks, tend to rely heavily on the least squares loss function. However, there are situations where such learning algorithms may not suffice, particularly in the presence of heavy-tailed noises or outliers. This predicament emphasizes the necessity of exploring alternative loss functions that can handle such scenarios more effectively, thereby unleashing the true potential of CNNs. The generalization error of deep CNNs is investigated with the rectified linear unit (ReLU) activation function for robust regression problems within an information-theoretic learning framework. It is demonstrated that when the regression function exhibits an additive ridge structure, and the noise possesses a finite $p$-th moment, the empirical risk minimization scheme, generated by the maximum correntropy criterion and deep CNNs, achieves fast convergence rates. Notably, these rates align with the mini-max optimal convergence rates attained by a fully connected neural network model with the Huber loss function up to a logarithmic factor.
