ODIN: An Object Detection and Instance Segmentation Diagnosis Framework

Object detection and instance segmentation are major tasks in Computer Vision and have substantially progressed after the introduction of Deep Convolutional Neural Network (DCNN). Analyzing the performance of DCNNs is an open research issue, addressed with attention techniques that inspect the response of inner network layers to input stimuli. A complementary approach relies on the black-box diagnosis of errors, which exploits ad hoc metadata on the input data set and factors the performance into indicators sensible or impacted by specific facets of the input (e.g., object size, presence of occlusions, image acquisition conditions, etc.). In this paper we present an open source error diagnosis framework for object detection and instance segmentation that helps model developers to add meta-annotations to their data sets, to compute performance metrics split by meta-annotation values, and to visualize diagnosis reports. The framework accepts the popular PASCAL VOC and MS COCO input formats, is agnostic to the training platform, and can be extended with application- and domain-specific meta-annotations and metrics with almost no coding.