A2020
Title: Enhancing the reliability of deep learning-based shoeprint image analysis via explainable AI (XAI)
Authors: Siwon Jeong - Pusan National University (Korea, South) [presenting]
Abstract: In forensics, shoeprints are vital for identification due to their Randomly Acquired Characteristics (RACs); however, their evaluation is often constrained by subjective judgment and a lack of quantitative metrics. Despite strong performance, the black-box nature of deep learning-based shoeprint analysis limits forensic interpretability and evidential reliability. A deep learning framework is proposed for shoeprint image analysis, integrating a Siamese network with explainable AI, transfer learning, and triplet-based metric learning with hard negative mining to improve discrimination between close non-matches under limited data. Two complementary XAI strategies are adopted: Grad-CAM localizes salient regions in the embedding network, whereas SINEX explains the final similarity score through a perturbation-based analysis that measures how masking segmented regions alters the Siamese output. The resulting heatmaps quantify the contribution of local visual evidence to each match decision. The framework jointly assesses predictive performance and explanation quality via ROC-AUC, EER, and Insertion/Deletion, alongside Localization Agreement (alignment with expert-identified RACs) and Pairwise Consistency (robustness of highlighted features across different image pairs). By achieving closer alignment with discriminative characteristics than existing methods, the framework provides a statistically grounded, reproducible basis for objective evidence assessment in judicial contexts.
