Partially online damage detection using long-term modal data under severe environmental effects by unsupervised feature selection and local metric learning

Distance-based anomaly detectors are among the most efficient unsupervised learning methods due to their non-parametric properties, inexpensive computational requirements, and simplicity. However, some major challenges including severe variability in data, low detectability, and a reliable decision threshold seriously affect overall performances of such techniques for long-term structural health monitoring (SHM). This article proposes a new distance-based anomaly detection method for partially online damage detection using the concepts of unsupervised feature selection and local metric learning. The main objective of unsupervised feature selection is to exploit one-class nearest neighbor search for extracting relevant features and removing irrelevant ones by a local Mahalanobis-squared distance (MSD). Because the choice of adequate nearest neighbors is critical to estimate local covariance matrices needed for distance metric learning, this issue is addressed by developing a hyperparameter optimization algorithm based on a statistical hypothesis test. An enhanced local MSD is also proposed to compute anomaly scores for decision-making. To estimate a reliable decision threshold, this article utilizes the peak-over-threshold technique under extreme value theory and generalized Pareto distribution. Due to the importance of selecting optimal extreme values and a probability rate of false alarm for threshold estimation, two hyperparameter optimization algorithms are designed to choose these unknown parameters. The major contribution of this article is to propose an innovative anomaly detector in conjunction with an enhanced multivariate distance under the concept of distance metric learning. Full-scale concrete and steel bridges under severe environmental variability are considered to validate the proposed method along with several comparisons. Results demonstrate that this method succeeds in long-term SHM under strong environmental variations in modal data and it also outperforms some well-known anomaly detection techniques in terms of effectiveness and efficiency.