Enhancing self-consumption ratio in a smart microgrid by applying a reinforcement learning-based energy management system

This study presents an updated version of the CityLearn Gym environment by integrating a stochastic data-driven vehicle-to-building model. To this end, EVs are modeled as local mobile storage using stochastic behavior derived from a real-world charging dataset, considering uncertainties in EV arrival/departure times, battery capacity, and the arrival state of charge (SoC). Then, the model is integrated within CityLearn to use a reinforcement learning-based energy management system (EMS) to control and optimize a smart microgrid's energy consumption and storage systems. A real-world microgrid in Norway is used to evaluate system performance under three scenarios, including one where solar panel (PV) generation is shared across buildings. The main objective is to provide energy flexibility by enhancing the self-energy consumption of solar generation by finding the optimal control policy for storage systems, which are batteries and EVs. The proposed EMS is designed using the soft actor-critic (SAC) algorithm to coordinate among the different flexible sources by defining the priority resources and direct charging control signals. Three scenarios are investigated and the shared scenario, which in PV generation can be shared between buildings, has had the best performance. The performance of the EMS is evaluated by five key indicators. The results show that the self-consumption ratio of microgrid has been increased up to 23 % and daily peak power has been reduced by up to 20 % compared to RBC as a conventional method. This highlights the impact of storage systems, especially EVs, on the microgrid performance to increase the penetration of solar energy through the energy transition and the potential of RL in advancing intelligent EMS design for future energy systems.