Addressing Temporal Dependence, Concept Drifts, and Forgetting in Data Streams

Several challenges arise when applying Machine Learning in data streams. Firstly, data points are continuously generated, and algorithms should continuously learn from each. The solution should also promptly adapt to changes in data distribution (known as concept drifts). Additionally, while addressing concept drifts, preserving the knowledge gained from past data is crucial to avoid the problem of catastrophic forgetting. Finally, one should consider the temporal dependence that data points may exhibit. Three communities address these problems separately: Streaming Machine Learning (SML), Continual Learning (CL), and Time Series Analysis (TSA). In our previous research, we proposed Continuous Progressive Neural Networks (cPNN), a first approach that considers all the challenges together. It bridges SML, CL, and TSA by producing a continuous adaptation of the CL strategy of Progressive Neural Networks. It uses transfer learning to adapt to changes quickly and manages temporal dependence using Long Short-Term Memory. In this work, we present a comprehensive experimental campaign that analyzes the behaviour of SML models and cPNN in the case of complex temporal dependence and various concept drifts on synthetic and real data streams. Results bring statistical evidence that SML models struggle with substantial temporal dependence, while cPNN is a viable solution.