Scalable Multi-Agent Reinforcement Learning for Dynamic Coordinated Multipoint Clustering

Fenghe Hu; Yansha Deng; A. Hamid Aghvami

doi:10.1109/TCOMM.2022.3220870

Scalable Multi-Agent Reinforcement Learning for Dynamic Coordinated Multipoint Clustering

Fenghe Hu, Yansha Deng^*, A. Hamid Aghvami

^*Corresponding author for this work

King's College London

Research output: Contribution to journal › Article › peer-review

15 Citations (Scopus)

Abstract

Reinforcement learning (RL) is a widely investigated intelligent algorithm and proved to be useful in the wireless communication area. However, for optimization problems in large-scale multi-cell networks whose dimension increases exponentially, it is unrealistic to employ a conventional centralized RL algorithm and make decisions for the entire network. Multi-agent RL, which allows distribute decision-making, is expected to solve the scalability problem but with performance issues due to the unknown global information, i.e., non-stationary environment. In this paper, we propose a parameter-sharing multi-agent RL for grouping decisions of coordinated multi-point in a large-scale network, where agents jointly serve users to enhance the cell-edge service. By sharing information via parameters, our theoretical and simulation results show that parameter sharing can largely benefit the multi-agent algorithm with convergence proof and convergence speed analysis. To reduce the effect of biased local heterogeneous experience, we also propose a transfer learning method for the parameter sharing process, whose performance of transfer learning algorithms is verified by the simulation results.

Original language	English
Pages (from-to)	101-114
Number of pages	14
Journal	IEEE TRANSACTIONS ON COMMUNICATIONS
Volume	71
Issue number	1
DOIs	https://doi.org/10.1109/TCOMM.2022.3220870
Publication status	Published - 1 Jan 2023

Keywords

multi-agent systems
next generation networking
Reinforcement learning
telecommunication

Access to Document

10.1109/TCOMM.2022.3220870

Cite this

@article{6b7994ca2f6d490a8146bf1145afe94b,

title = "Scalable Multi-Agent Reinforcement Learning for Dynamic Coordinated Multipoint Clustering",

abstract = "Reinforcement learning (RL) is a widely investigated intelligent algorithm and proved to be useful in the wireless communication area. However, for optimization problems in large-scale multi-cell networks whose dimension increases exponentially, it is unrealistic to employ a conventional centralized RL algorithm and make decisions for the entire network. Multi-agent RL, which allows distribute decision-making, is expected to solve the scalability problem but with performance issues due to the unknown global information, i.e., non-stationary environment. In this paper, we propose a parameter-sharing multi-agent RL for grouping decisions of coordinated multi-point in a large-scale network, where agents jointly serve users to enhance the cell-edge service. By sharing information via parameters, our theoretical and simulation results show that parameter sharing can largely benefit the multi-agent algorithm with convergence proof and convergence speed analysis. To reduce the effect of biased local heterogeneous experience, we also propose a transfer learning method for the parameter sharing process, whose performance of transfer learning algorithms is verified by the simulation results.",

keywords = "multi-agent systems, next generation networking, Reinforcement learning, telecommunication",

author = "Fenghe Hu and Yansha Deng and {Hamid Aghvami}, A.",

note = "Publisher Copyright: {\textcopyright} 1972-2012 IEEE.",

year = "2023",

month = jan,

day = "1",

doi = "10.1109/TCOMM.2022.3220870",

language = "English",

volume = "71",

pages = "101--114",

journal = "IEEE TRANSACTIONS ON COMMUNICATIONS",

issn = "0090-6778",

publisher = "I E E E",

number = "1",

}

TY - JOUR

T1 - Scalable Multi-Agent Reinforcement Learning for Dynamic Coordinated Multipoint Clustering

AU - Hu, Fenghe

AU - Deng, Yansha

AU - Hamid Aghvami, A.

PY - 2023/1/1

Y1 - 2023/1/1

N2 - Reinforcement learning (RL) is a widely investigated intelligent algorithm and proved to be useful in the wireless communication area. However, for optimization problems in large-scale multi-cell networks whose dimension increases exponentially, it is unrealistic to employ a conventional centralized RL algorithm and make decisions for the entire network. Multi-agent RL, which allows distribute decision-making, is expected to solve the scalability problem but with performance issues due to the unknown global information, i.e., non-stationary environment. In this paper, we propose a parameter-sharing multi-agent RL for grouping decisions of coordinated multi-point in a large-scale network, where agents jointly serve users to enhance the cell-edge service. By sharing information via parameters, our theoretical and simulation results show that parameter sharing can largely benefit the multi-agent algorithm with convergence proof and convergence speed analysis. To reduce the effect of biased local heterogeneous experience, we also propose a transfer learning method for the parameter sharing process, whose performance of transfer learning algorithms is verified by the simulation results.

AB - Reinforcement learning (RL) is a widely investigated intelligent algorithm and proved to be useful in the wireless communication area. However, for optimization problems in large-scale multi-cell networks whose dimension increases exponentially, it is unrealistic to employ a conventional centralized RL algorithm and make decisions for the entire network. Multi-agent RL, which allows distribute decision-making, is expected to solve the scalability problem but with performance issues due to the unknown global information, i.e., non-stationary environment. In this paper, we propose a parameter-sharing multi-agent RL for grouping decisions of coordinated multi-point in a large-scale network, where agents jointly serve users to enhance the cell-edge service. By sharing information via parameters, our theoretical and simulation results show that parameter sharing can largely benefit the multi-agent algorithm with convergence proof and convergence speed analysis. To reduce the effect of biased local heterogeneous experience, we also propose a transfer learning method for the parameter sharing process, whose performance of transfer learning algorithms is verified by the simulation results.

KW - multi-agent systems

KW - next generation networking

KW - Reinforcement learning

KW - telecommunication

UR - http://www.scopus.com/inward/record.url?scp=85141623316&partnerID=8YFLogxK

U2 - 10.1109/TCOMM.2022.3220870

DO - 10.1109/TCOMM.2022.3220870

M3 - Article

AN - SCOPUS:85141623316

SN - 0090-6778

VL - 71

SP - 101

EP - 114

JO - IEEE TRANSACTIONS ON COMMUNICATIONS

JF - IEEE TRANSACTIONS ON COMMUNICATIONS

IS - 1

ER -

Scalable Multi-Agent Reinforcement Learning for Dynamic Coordinated Multipoint Clustering

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this