TY - JOUR
T1 - Cooperative Multigroup Broadcast 360° Video Delivery Network
T2 - A Hierarchical Federated Deep Reinforcement Learning Approach
AU - Hu, Fenghe
AU - Deng, Yansha
AU - Aghvami, A. Hamid
N1 - Publisher Copyright:
© 2002-2012 IEEE.
PY - 2022/6/1
Y1 - 2022/6/1
N2 - With the stringent requirement of receiving video from the unmanned aerial vehicle (UAV) from anywhere in the stadium of sports events and the significant-high per-cell throughput for video transmission to virtual reality (VR) users, a promising solution is a cell-free multi-group broadcast (CF-MB) network with cooperative reception and broadcast access-points (AP). To explore the benefit of broadcasting user-correlated decode-dependent video resources to spatially correlated VR users, the network should dynamically schedule the video and cluster APs into virtual cells for a different group of VR users with overlapped video requests. By decomposing the problem into scheduling and association sub-problems, we first introduce the conventional non-learning-based scheduling and association algorithms, and a centralized deep reinforcement learning (DRL) association approach based on the rainbow agent with a convolutional neural network (CNN) to generate decisions from observation. To reduce its complexity, we then decompose the association problem into multiple sub-problems, resulting in a networked-distributed Partially Observable Markov decision process (ND-POMDP). To solve it, we propose a multi-agent deep DRL algorithm. To jointly solve the coupled association and scheduling problems, we further develop a hierarchical federated DRL algorithm with scheduler as meta-controller, and association as the controller. Our simulation results show that our CF-MB network can effectively handle real-time video transmission from UAVs to VR users. Our proposed learning architecture is effective and scalable for a high-dimensional cooperative association problem with increasing APs and VR users. Also, our proposed algorithms outperform non-learning based methods with significant performance improvement.
AB - With the stringent requirement of receiving video from the unmanned aerial vehicle (UAV) from anywhere in the stadium of sports events and the significant-high per-cell throughput for video transmission to virtual reality (VR) users, a promising solution is a cell-free multi-group broadcast (CF-MB) network with cooperative reception and broadcast access-points (AP). To explore the benefit of broadcasting user-correlated decode-dependent video resources to spatially correlated VR users, the network should dynamically schedule the video and cluster APs into virtual cells for a different group of VR users with overlapped video requests. By decomposing the problem into scheduling and association sub-problems, we first introduce the conventional non-learning-based scheduling and association algorithms, and a centralized deep reinforcement learning (DRL) association approach based on the rainbow agent with a convolutional neural network (CNN) to generate decisions from observation. To reduce its complexity, we then decompose the association problem into multiple sub-problems, resulting in a networked-distributed Partially Observable Markov decision process (ND-POMDP). To solve it, we propose a multi-agent deep DRL algorithm. To jointly solve the coupled association and scheduling problems, we further develop a hierarchical federated DRL algorithm with scheduler as meta-controller, and association as the controller. Our simulation results show that our CF-MB network can effectively handle real-time video transmission from UAVs to VR users. Our proposed learning architecture is effective and scalable for a high-dimensional cooperative association problem with increasing APs and VR users. Also, our proposed algorithms outperform non-learning based methods with significant performance improvement.
KW - Digital video broadcasting
KW - machine learning
KW - reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85132336952&partnerID=8YFLogxK
U2 - 10.1109/TWC.2021.3126147
DO - 10.1109/TWC.2021.3126147
M3 - Article
AN - SCOPUS:85132336952
SN - 1536-1276
VL - 21
SP - 4009
EP - 4024
JO - IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
JF - IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
IS - 6
ER -