Activation-Based Concept Extraction for Explainability in Image Classification

Concept-based explainability aims to interpret deep learning models such as CNNs in terms of high-level, human-understandable concepts. However, the need to manually define concepts often limits these approaches, as it requires domain knowledge and manual effort to collect example images for each concept. Moreover, many automatic extractors rely on random cropping or activation-agnostic segmentation, which can produce many noisy or irrelevant concepts. To address this, we introduce Activation-Based Concepts (ABC), a post-hoc technique for automatically extracting visual concepts aligned with the model's internal activations, whose importance for a prediction can be computed using standard concept attribution methods. To validate the proposed approach, we perform a user study to measure concept understandability and coherence, and a concept-removal experiment to assess fidelity. We compare ABC with other state-of-the-art methods for post-hoc concept extraction, showing better understandability and concept coherence with comparable fidelity.