A Review of Agent-Based Models for Energy Commodity Markets and Their Natural Integration with RL Models

Agent-based models are a flexible and scalable modeling approach employed to study and describe the evolution of complex systems in different fields, such as social sciences, engineering, and economics. In the latter, they have been largely employed to model financial markets with a bottom-up approach, with the aim of understanding the price formation mechanism and to generate market scenarios. In the last few years, they have found application in the analysis of energy markets, which have experienced profound transformations driven by the introduction of energy policies to ease the penetration of renewable energy sources and the integration of electric vehicles and by the current unstable geopolitical situation. This review provides a comprehensive overview of the application of agent-based models in energy commodity markets by defining their characteristics and highlighting the different possible applications and the open-source tools available. In addition, it explores the possible integration of agent-based models with machine learning techniques, which makes them adaptable and flexible to the current market conditions, enabling the development of dynamic simulations without fixed rules and policies. The main findings reveal that while agent-based models significantly enhance the understanding of energy market mechanisms, enabling better profit optimization and technical constraint coherence for traders, scaling these models to highly complex systems with a large number of agents remains a key limitation.