Unsupervised data normalization for continuous dynamic monitoring by an innovative hybrid feature weighting-selection algorithm and natural nearest neighbor searching

Continuous dynamic monitoring brings an important opportunity to evaluate the health and integrity of civil structures in a long-term manner. However, high dimensionality and sparsity of data caused by long-term monitoring and negative influences of environmental and/or operational variability are major challenges in this process. To address these important issues, this article proposes an innovative unsupervised data normalization method based on a novel hybrid feature weighting-selection algorithm and the idea of natural nearest neighbor (NN) searching emanated from the theory of mutual friendships in human societies. The proposed hybrid algorithm is a combination of global feature weighting with a new weighting measure and local feature selection. For this algorithm, this article leverages the natural NN searching that seeks to find adequate NNs automatically. The main objective of the proposed method is to remove the environmental and/or operational effects and provide normalized weighted features for reliable continuous dynamic monitoring. Using such features, an anomaly detector based on the Mahalanobis-squared distance is developed to assess and detect structural damage. The key innovations of this paper contain proposing a fully nonparametric unsupervised learning technique in two parts of data normalization and anomaly detection and developing a novel hybrid algorithm for removing the environmental and/or operational variations. Long-term dynamic features (modal frequencies) of a three-span box-girder concrete bridge (Z24 Bridge) and a long-span concrete arch bridge (Infante Dom Henrique Bridge) are considered to verify the proposed technique with several comparisons. Results indicate that this technique is successful and reliable in mitigating the environmental and/or operational effects and notifying accurate structural states.