publications

2023

1. AAMAS

   SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control

   Harsh Goel, Yifeng Zhang, Mehul Damani, and 1 more author

   In Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems, 2023

   Abs PDF

   Many recent works have turned to multi-agent reinforcement learn- ing (MARL) for adaptive traffic signal control to optimize the travel time of vehicles over large urban networks. However, achieving ef- fective and scalable cooperation among junctions (agents) remains an open challenge, as existing methods often rely on extensive, non- generalizable reward shaping or on non-scalable centralized learn- ing. To address these problems, we propose a new MARL method for traffic signal control, SocialLight, which learns cooperative traffic control policies by distributedly estimating the individual marginal contribution of agents on their local neighborhood. SocialLight re- lies on the Asynchronous Actor Critic (A3C) framework, and makes learning scalable by learning a locally-centralized critic conditioned over the states and actions of neighboring agents, used by agents to estimate individual contributions by counterfactual reasoning. We further introduce important modifications to the advantage calculation that help stabilize policy updates. These modifications decouple the impact of the neighbors’ actions on the computed advantages, thereby reducing the variance in the gradient updates. We benchmark our trained network against state-of-the-art traf- fic signal control methods on standard benchmarks in two traffic simulators, SUMO and CityFlow. Our results show that SocialLight exhibits improved scalability to larger road networks and better performance across usual traffic metrics.

2022

1. IROS - AI2R

   Reinforcement Learning for Agile Active Target Sensing with a UAV

   Harsh Goel, Laura Jarin Lipschitz, Saurav Agarwal, and 2 more authors

   2022