Title: Towards reverse algorithmic engineering of neural networks Authors:  Dan Vilenchik - Ben-Gurion University (Israel) [presenting]
Abstract: As machine learning models get more complex, they can outperform traditional algorithms and tackle a wider range of problems, including challenging combinatorial optimization tasks. However, this increased complexity can make understanding how the model makes its decisions difficult. Explainable models can increase trust in the models' decisions and may even lead to improvements in the algorithm itself. Algorithms like GradCAM or SHAP provide good explanations in terms of feature importance, typically for classification tasks, but they provide little insight when the ML pipeline is designed to work, for example, as an algorithm for solving optimization problems. A framework for explaining a model's decision-making process is presented from an algorithmic point of view while taking into account domain knowledge of the problem at hand. Using the NeuroSAT algorithm for SAT solving as a case study, it is demonstrated how the framework explains the underlying algorithmic concepts that drive the operation of an NN-based model. For example, it is discovered that for sparse random SAT instances, NeuroSAT mimics the pure literal heuristic, while for denser formulas, it relies on the concept of support to decide which variables to flip, similar to the WalkSAT algorithm.