Reinforcement Distribution in Fuzzy Q-Learning

Q-learning is one of the most popular reinforcement learning methods that allows an agent to learn the relationship between interval-valued state and action spaces, through a direct interaction with the environment. Fuzzy Q-learning is an extension to this algorithm to enable it to evolve fuzzy inference systems (FIS) which range on continuous state and action spaces. In a FIS, the interaction among fuzzy rules plays a primary role to achieve good performance and robustness. Learning a system where this interaction is present gives to the learning mechanism problems due to eventually incoherent reinforcements coming to the same rule due to its interaction with other rules. In this paper, we will introduce different strategies to distribute reinforcement to reduce this undesired effect and to stabilize the obtained reinforcement. In particular, we will present two strategies: the former focuses on rewarding the actions chosen by each rule during the cooperation phase, the latter on rewarding the rules presenting actions closer to those actually executed rather than the rules that contributed to generate such actions.