Fairness-aware Multi-agent Reinforcement Learning and Visual Perception For Adaptive Traffic Signal Control

Abstract:

The majority of Multi-agent Reinforcement Learning (MARL) methods for solving Adaptive Traffic Signal Control (ATSC) problems are dedicated to maximizing the throughput while ignoring fairness, resulting in a bad situation where some vehicles keep waiting. For this reason, this paper models the ATSC problem as a Partially Observable Markov Game (POMG), in which a value function that combines throughput and fairness is elaborated. On this basis, we propose a new cooperative MARL method FA-MAPPO, i.e., fairness-aware multi-agent proximity policy optimization, which is based on the cooperative MARL algorithm MAPPO. In addition, FA-MAPPO uses graph attention neural networks to efficiently extract state representations from traffic data acquired through visual perception in multi-intersection scenarios. Experimental results in Jinan and synthetic scenarios confirm that FA-MAPPO improves fairness while guaranteeing passage efficiency compared to the SOTA methods.

Key words:Multi-agent Reinforcement Learning, Visual Perception, Adaptive Traffic Signal Control