A1859
Title: PCA-based layer compression for Siamese neural networks
Authors: Elizabeth Chou - National Chengchi University (Taiwan) [presenting]
Abstract: Siamese neural networks are widely used in similarity learning and pairwise classification, especially when paired observations provide useful supervision. However, their hidden layers may contain redundant neurons, increasing model complexity and computational cost. A principal component analysis (PCA)-based layer compression approach for Siamese neural networks is proposed. The method extracts the weight matrix of a target fully connected layer, applies PCA to its weight structure, and replaces the original layer with a lower-dimensional layer formed from the retained principal components. Thus, the compressed model is obtained through PCA-induced layer replacement rather than direct neuron pruning. The proposed method is interpreted from a low-rank approximation perspective. The retained principal components preserve dominant weight variation, while the explained-variance threshold controls the tradeoff between compression strength and information preservation. Empirical studies on both image and tabular datasets indicate that PCA-compressed models can reduce the number of neurons while maintaining, and in some cases improving, pairwise classification performance. These results suggest that PCA-based layer compression provides a simple and interpretable strategy for balancing model simplicity and predictive accuracy in Siamese representation learning.
