Inferring human preferences in multisector systems via Inverse Reinforcement Learning

In a changing climate and society, the importance of cross-sector interactions becomes crucial for understanding the co-evolution of human and natural systems, where the role of individual and collective human decisions is a major driver of system vulnerabilities and adaptive capacity. While mathematical models of natural processes have been studied and developed for centuries and, today, they are extremely sophisticated at fine spatial and temporal scales, there is an urgent need to shed light on the key role of human behaviors across multisector systems. For instance, the majority of current global hydrological models incorporate pre-defined rules for simulating reservoir operations, which distinguish between reservoirs used for irrigation or non-irrigation purposes only. However, many water systems are operated to meet competing multi-sector demands and it is often unclear how operators confront these demands. In this work, we introduce a Reinforcement Learning approach to model the dynamics of multipurpose reservoir systems. Specifically, our method first uses Inverse Reinforcement Learning to identify the trade-off among competing objectives from historical observations of the reservoir system dynamics. The identified objective function is then used in the formulation of an optimal control problem returning a closed-loop policy which allows the simulation of the observed dynamics of the reservoir system. We demonstrate the potential of the proposed method in a real-world application involving the multipurpose regulation of Lake Como in northern Italy. Results show that our approach effectively infers the trade-off between flood control and water supply adopted in the observed system’s operation, and yields a control policy that closely approximates the observed system dynamics.