Unsupervised discovery of Interpretable Visual Concepts. (arXiv:2309.00018v1 [cs.CV])

Providing interpretability of deep-learning models to non-experts, while
fundamental for a responsible real-world usage, is challenging. Attribution
maps from xAI techniques, such as Integrated Gradients, are a typical example
of a visualization technique containing a high level of information, but with
difficult interpretation. In this paper, we propose two methods, Maximum
Activation Groups Extraction (MAGE) and Multiscale Interpretable Visualization
(Ms-IV), to explain the model’s decision, enhancing global interpretability.
MAGE finds, for a given CNN, combinations of features which, globally, form a
semantic meaning, that we call concepts. We group these similar feature
patterns by clustering in “concepts”, that we visualize through Ms-IV. This
last method is inspired by Occlusion and Sensitivity analysis (incorporating
causality), and uses a novel metric, called Class-aware Order Correlation
(CaOC), to globally evaluate the most important image regions according to the
model’s decision space. We compare our approach to xAI methods such as LIME and
Integrated Gradients. Experimental results evince the Ms-IV higher localization
and faithfulness values. Finally, qualitative evaluation of combined MAGE and
Ms-IV demonstrate humans’ ability to agree, based on the visualization, on the
decision of clusters’ concepts; and, to detect, among a given set of networks,
the existence of bias.