Multi-State End-to-End Learning for Autonomous Vehicle Lateral Control

Lateral control is one of the primary requirements of an autonomous vehicle. This task is generally performed using complex pipelines, which include line detection trough neural network processing, vehicle state estimation, and planning. What we propose in this paper is an alternative end-to-end approach to the problem. Images acquired by a camera mounted on the vehicle are processed by two convolutional neural networks to directly retrieve the steering command. In particular, we propose an architecture built using two connected neural networks, one to predict the scenario the vehicle is facing and one, conditioned on possible situations, to predict the steering command. In our work, we also analyze the potential of a computer-generated dataset for a demanding task like end-to-end learning, where the image quality is fundamental. All the training is then performed on synthetic images, while the testing is done on real data acquired by an experimental vehicle.