Adaptive Model Pruning for Hierarchical Wireless Federated Learning

TY - CHAP

T1 - Adaptive Model Pruning for Hierarchical Wireless Federated Learning

AU - Liu, Xiaonan

AU - Wang, Shiqiang

AU - Deng, Yansha

AU - Nallanathan, Arumugam

N1 - Publisher Copyright: © 2024 IEEE.

PY - 2024

Y1 - 2024

N2 - Federated Learning (FL) is a promising privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. Hierarchical FL (HFL), as a device-edge-cloud aggregation hierarchy, can enjoy both the cloud server's access to more datasets and the edge servers' efficient communications with devices. However, the learning latency increases with the HFL network scale due to the increasing number of edge servers and devices with limited local computation capability and communication bandwidth. To address this issue, in this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We present the convergence rate of an upper on the l2-norm of gradients for HFL with model pruning, analyze the computation and communication latency of the proposed model pruning scheme, and formulate an optimization problem to maximize the convergence rate under a given latency threshold by jointly optimizing the pruning ratio and wireless resource allocation. By decoupling the optimization problem and using Karush-Kuhn-Tucker (KKT) conditions, closed-form solutions of pruning ratio and wireless resource allocation are derived. Simulation results show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50% communication cost.

AB - Federated Learning (FL) is a promising privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. Hierarchical FL (HFL), as a device-edge-cloud aggregation hierarchy, can enjoy both the cloud server's access to more datasets and the edge servers' efficient communications with devices. However, the learning latency increases with the HFL network scale due to the increasing number of edge servers and devices with limited local computation capability and communication bandwidth. To address this issue, in this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We present the convergence rate of an upper on the l2-norm of gradients for HFL with model pruning, analyze the computation and communication latency of the proposed model pruning scheme, and formulate an optimization problem to maximize the convergence rate under a given latency threshold by jointly optimizing the pruning ratio and wireless resource allocation. By decoupling the optimization problem and using Karush-Kuhn-Tucker (KKT) conditions, closed-form solutions of pruning ratio and wireless resource allocation are derived. Simulation results show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50% communication cost.

KW - communication and computation latency

KW - federated pruning

KW - Hierarchical Wireless network

KW - machine learning

UR - http://www.scopus.com/inward/record.url?scp=85198834178&partnerID=8YFLogxK

U2 - 10.1109/WCNC57260.2024.10571275

DO - 10.1109/WCNC57260.2024.10571275

M3 - Conference paper

AN - SCOPUS:85198834178

BT - 2024 IEEE Wireless Communications and Networking Conference, WCNC 2024 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 25th IEEE Wireless Communications and Networking Conference, WCNC 2024

Y2 - 21 April 2024 through 24 April 2024

ER -