Advancing Forecast Informed Reservoir Operations via Neuro-Evolutionary Direct Policy Search

Recent advances in hydroclimatic forecasting at various temporal and spatial scales offer new opportunities to support water reservoir operators in coping with the acceleration of the hydrologic cycle. However, selecting the best forecast information and assimilating it into operating policies to generate an added value across diverse operating objectives is still a challenging task. Here, we explore the potential of adopting a Reinforcement Learning (RL) framework for supporting forecast informed reservoir operations. Our RL framework is built on the Neuro-Evolutionary Multi-Objective Direct Policy Search (NEMODPS) algorithm, which uses evolutionary strategies to identify the best performing policy architectures (e.g., nonlinear approximating network with diverse activation functions and variable numbers of hidden layers) and associated input variables. This framework is demonstrated using the Lake Como system in Northern Italy, a regulated lake primarily operated for flood control and water supply. The sub-alpine basin of the lake is characterized by mixed slow and fast dynamics resulting from the snow- and rain-dominated hydrology. In this context, different forecasts over short and seasonal time scales are available, including short-term (i.e., 60 hours lead time) deterministic forecasts produced with locally calibrated models as well as the sub-seasonal and seasonal forecasts of the Copernicus Emergency Management Service's European Flood Awareness System. Preliminary results suggest the our RL framework is able to design Pareto-optimal operating policies that leverage the flexibility of the search process for specializing both the policy architectures and inputs when navigating the multisectoral tradeoffs associated with the Lake Como regulation.