Optimal control of compartmental models: The exact solution

We formulate a control problem for positive compartmental systems formed by nodes (buffers) and arcs (flows). Our main result is that, on a finite horizon, we can solve the Pontryagin equations in one shot without resorting to trial and error via shooting. As expected, the solution is bang-bang and the switching times can be easily determined. We are also able to find a cost-to-go-function, in an analytic form, by solving a simple nonlinear differential equation. On an infinite horizon, we consider the Hamilton-Jacobi-Bellman theory and we show that the HJB equation can be solved exactly. Moreover, we show that the optimal solution is constant and the cost-to-go function is linear and copositive. This function is the solution of a nonlinear equation. We propose an iterative scheme for solving this equation, which converges in finite time. We also show that an exact solution can be found if there is a positive external disturbance affecting the process and the problem is formulated in a min sup framework. We finally provide illustrative examples related to flood control and epidemiology.