Neuro-Evolutionary Direct Policy Search for the optimal control of multi-purpose water resources systems

The problem of designing a control policy for water resources systems is characterized by a complex decision space, a noisy environment, and the coexistence of multiple control objectives. Direct Policy Search (DPS) approach is among the most widely applied algorithms for these problems. In DPS, the control policy is defined within a functional class, and the optimal parameterization is searched with an optimization tool. DPS can thus find, at most, the best solution within the selected class of functions. While many works focused on improving the optimization component, very few tackled the issue of selecting an optimal policy architecture. In this work, we present a novel policy search routine called Neuro-EVOlutionary Direct Policy Search (NEVODPS), which conjunctively searches the architectural and parameter space to select a tradeoff-dependent optimal functional class, along with the associated optimal parameterization. We tested NEVODPS on the Lake Como case study, a regulated lake located in Northern Italy, which is operated trading off irrigation supply and flood control. Numerical results show that the Pareto-dynamic structural and parametrical policy search of NEVODPS outperforms the solutions designed via traditional DPS with predefined policy topologies.