A Markov Decision Process framework to incorporate network-level data in motion planning for connected and automated vehicles

Autonomy and connectivity are expected to enhance safety and improve fuel efficiency in transportation systems. While connected vehicle-enabled technologies, such as coordinated cruise control, can improve vehicle motion planning by incorporating information beyond the line of sight of vehicles, their benefits are limited by the current short-sighted planning strategies that only utilize local information. In this paper, we propose a framework that devises vehicle trajectories by coupling a locally-optimal motion planner with a Markov decision process (MDP) model that can capture network-level information. Our proposed framework can guarantee safety while minimizing a trip's generalized cost, which comprises of its fuel and time costs. To showcase the benefits of incorporating network-level data when devising vehicle trajectories, we conduct a comprehensive simulation study in three experimental settings, namely a circular track, a highway with on- and off-ramps, and a small urban network. The simulation results indicate that statistically significant efficiency can be obtained for the subject vehicle and its surrounding vehicles in different traffic states under all experimental settings. This paper serves as a proof-of-concept to showcase how connectivity and autonomy can be leveraged to incorporate network-level information into motion planning.