Aperiodic speed tracking control of unmanned sightseeing vehicles based on hierarchical reinforcement learning

This paper proposes an aperiodic control approach based on hierarchical reinforcement learning for the longitudinal speed tracking control of unmanned sightseeing vehicles in scenic areas. The method develops the MAXQ hierarchical framework to decompose the speed tracking task into a series of subtasks, including learning the control inputs and the triggering instants. This allows each subtask to focus on a specific local issue, independently learning and optimising, making the learning process simpler and more efficient. In the learning of triggering instants, an event-triggered strategy is incorporated, which greatly reduces the controller update rate, thereby reducing communication and resources occupation. In the learning of control inputs, the reward function not only considers tracking performance, but also considers both inter-vehicle distance and acceleration change rate, which ensures tracking accuracy while enhancing driving safety and comfort. Finally, the efficacy of the proposed method is validated through a simulation study.