Reinforcement Learning Based Advertising Strategy Using Crowdsensing Vehicular Data

Lou, Kaihao, Yang, Yongjian, Wang, En, Liu, Zheli, Baker, Thar and Bashir, Ali Kashif ORCID: https://orcid.org/0000-0001-7595-2522 (2021) Reinforcement Learning Based Advertising Strategy Using Crowdsensing Vehicular Data. IEEE Transactions on Intelligent Transportation Systems, 22 (7). pp. 4635-4647. ISSN 1524-9050

Preview

Accepted Version Available under License In Copyright. Download (1MB) | Preview

Abstract

As an effective tool, roadside digital billboard advertising is widely used to attract potential customers (e.g., drivers and passengers passing by the billboards) to obtain commercial profit for the advertiser, i.e., the attracted customers' payment. The commercial profit depends on the number of attracted customers, hence the advertiser needs to adopt an effective advertising strategy to determine the advertisement switching policy for each digital billboard to attract as many potential customers as possible. Whether a customer could be attracted is influenced by numerous factors, such as the probability that the customer could see the billboard and the degree of his/her interests in the advertisement. Besides, cooperation and competition among all digital billboards will also affect the commercial profit. Taking the above factors into consideration, we formulate the dynamic advertising problem to maximize the commercial profit for the advertiser. To address the problem, we first extract potential customers' implicit information by using the vehicular data collected by Mobile CrowdSensing (MCS), such as their vehicular trajectories and their preferences. With this information, we then propose an advertising strategy based on multi-agent deep reinforcement learning. By using the proposed advertising strategy, the advertiser could determine the advertising policy for each digital billboard and maximize the commercial profit. Extensive experiments on three real-world datasets have been conducted to verify that our proposed advertising strategy could achieve the superior commercial profit compared with the state-of-the-art strategies.