A Benchmark Suite to Evaluate DNN's Resilience

Assessing AI systems reliability is essential before deploying them in safety-critical applications. While recent efforts have focused on improving model resilience to random hardware faults, meaningful comparison remains difficult due to the lack of standardized reference models. Different authors use different implementations, which makes comparisons unfair and biased: resilience is influenced by the training processes, the software framework, and data representations. To address these issues, this work introduces a benchmark suite of CNN models to test the resilience of DNNs. The benchmark is structured on different axes: software framework, hardware platform, data representation, task and dataset. It is aimed at providing a shared foundation for fair and reproducible resilience evaluation.