King's College London

Research portal

A Reinforcement Learning-Based User-Assisted Caching Strategy for Dynamic Content Library in Small Cell Networks

Research output: Contribution to journalArticle

Xinruo Zhang, Gan Zheng, Sangarapillai Lambotharan, Mohammad Nakhai, Kai-Kit Wong

Original languageEnglish
Article number9020168
Pages (from-to)3627-3639
Number of pages13
JournalIEEE Transactions on Communications
Issue number6
Early online date2 Mar 2020
Accepted/In press23 Feb 2020
E-pub ahead of print2 Mar 2020
PublishedJun 2020


  • TCOM_Caching

    TCOM_Caching.pdf, 1.16 MB, application/pdf

    Uploaded date:27 Feb 2020

    Version:Accepted author manuscript

King's Authors


This paper studies the problem of joint edge cache placement and content delivery in cache-enabled small cell networks in the presence of spatio-temporal content dynamics unknown a priori. The small base stations (SBSs) satisfy users' content requests either directly from their local caches, or by retrieving from other SBSs' caches or from the content server. In contrast to previous approaches that assume a static content library at the server, this paper considers a more realistic non-stationary content library, where new contents may emerge over time at different locations. To keep track of spatio-temporal content dynamics, we propose that the new contents cached at users can be exploited by the SBSs to timely update their flexible cache memories in addition to their routine off-peak main cache updates from the content server. To take into account the variations in traffic demands as well as the limited caching space at the SBSs, a user-assisted caching strategy is proposed based on reinforcement learning principles to progressively optimize the caching policy with the target of maximizing the weighted network utility in the long run. Simulation results verify the superior performance of the proposed caching strategy against various benchmark designs.

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454