Curses, tradeoffs, and scalable management: advancing evolutionary multi-objective direct policy search to improve water reservoir operations

Optimal management policies for water reservoir operation are generally designed via stochastic dynamic programming (SDP), which can be applied under very mild assumptions met in the majority of water resources systems. Yet, the adoption of SDP in complex real-world problems is challenged by three curses that considerably limit its practical application: (i) the curse of dimensionality, i.e. the computational cost grows exponentially with state, decision, and disturbance dimension; (ii) the curse of modeling, i.e. any variable considered among the policy arguments has to be modeled; (iii) the curse of multiple objectives, i.e. the computational cost grows exponentially with the number of objectives considered. Simulation-based optimization represents a promising alternative, where the key idea is to parameterize the operating policy within a given family of functions and, then, to optimize the policy parameters with respect to the objectives of the problem. These approaches have been already explored in the literature with traditional operating rules (e.g., hedging rules for flood control) and single-objective, gradient-descent optimization algorithms. However, the effectiveness of these methods decreases when the complexity of the problems grows, as they require searching in a high-dimensional parameter space associated to noisy, non-linear, multi-modal objective functions. This study proposes the use of evolutionary multi-objective direct policy search (EMODPS) to design the operation of multi-purpose water reservoirs and develops a diagnostic framework to evaluate the quality of the Pareto optimal solutions in terms of the overall policies’ performance, the characteristics of the resulting Pareto fronts, and the reliability of the policy design process. The multi-purpose HoaBinh water reservoir in Vietnam, accounting for hydropower production and flood control, is used as a case study. To ensure the possibility of approximating the unknown optimal solution of the problem, we adopted two widely used nonlinear approximating networks, namely Artificial Neural Networks and Radial Basis Functions. Results show that EMODPS successfully scales to high-dimensional multi-objective problems, computing an approximation of the entire Pareto front in a single optimization run. The comparative analysis of ANN and RBF policy parameterizations suggests the general superiority of RBF over ANN. In addition, the RBF policies outperform the solutions designed via SDP, demonstrating the potential of EMODPS for advancing multi-objective water reservoir management.