Image Retrieval in Semiconductor Manufacturing

Content-Based Image Retrieval has a lot of applications in the industry, where large collections of data from manufacturing need to be automatically queried e.g. for quality inspection purposes. In this work we design an image retrieval solution over IMAGO, a dataset of Transmission Electron Microscopy (TEM) images of nano-sized silicon structures collected in the production site of STMicroelectronics, in Agrate Brianza, Italy. Image retrieval in imago is challenging because: i) only a limited portion of images are provided with labels, namely type of semiconductor structure, ii) most images refer to unseen classes that are not represented in the training set, and iii) images of the same class can be acquired at different magnification levels of the electronic microscope. Our main contribution is the design of a deep-learning based image retrieval system that leverages a training procedure that alternates between siamese loss, assessed on annotated samples, and reconstruction loss, assessed on unlabelled samples. Our solution exploits the whole information in the IMAGO dataset, and our experiments confirm we can successfully retrieve images of unseen classes that exhibit the same structure of the query ones. Our solution is currently deployed in STMicroelectronics production sites.