Enhancing Predictability in Deep Reinforcement Learning for Building Temperature Control

In recent years, machine learning-based methods have become increasingly popular for addressing control problems. Within the context of temperature control in buildings, reinforcement learning algorithms stand out as an appealing model-free and fast real-time execution alternative to classical model-based control methods. However, these techniques lack interpretability and require a significant amount of data. In this paper, we investigate the application of an actor-critic reinforcement learning algorithm for the temperature control of a medium-size building through a specific model-based training approach. More specifically, the goal is here to enhance the predictive capabilities of actor-critic schemes by proposing extensions to state-of-the-art algorithms, which integrate a simplified model as a state predictor and incorporate future disturbances. This leads to the so-called Recursive Actor-Critic, Look-Ahead Trajectory Actor-Critic, and Augmented Actor-Critic schemes. Experimental results show that such improved reinforcement learning approaches can achieve performance comparable to model predictive control, without the need for real-time optimization nor a huge amount of data.