Device Scheduling for Wireless Federated Learning with Latency and Representativity

TY - CONF

T1 - Device Scheduling for Wireless Federated Learning with Latency and Representativity

AU - Chen, Zhixiong

AU - Yi, Wenqiang

AU - Deng, Yansha

AU - Nallanathan, Arumugam

N1 - Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - Existing device scheduling methods in wireless fed-erated learning (FL) mainly focused on selecting the devices with maximum gradient norm or loss function and requires all devices to perform local training in each round. This may produce extra training costs and schedule devices with similar data statistics, thus degrading learning performance. To mitigate these prob-lems, we proposed to schedule a subset of representative devices and find the corresponding pre-device stepsizes to approximate the full participation aggregated gradient. Taking into account the limited wireless bandwidth, we formulate an optimization problem to capture the trade-off between representativity and latency by optimizing device scheduling and bandwidth allocation policies. Our analysis reveals optimal bandwidth allocation is achieved when all scheduled devices have the same latency consisting of computation and communication latencies. Then, by proving the non-monotone sub modularity of the problem, we develop a double greedy algorithm to solve the device scheduling policy. To avoid the local training of unscheduled devices, we utilize the historical gradient information of devices to estimate the current gradient for device scheduling design. Experimental results show that the proposed latency- and representativity-aware scheduling algorithm saves over 16% and 12% training time for MNIST and CIFAR-I0 datasets than the scheduling algorithms based on either latency and representativity individually.

AB - Existing device scheduling methods in wireless fed-erated learning (FL) mainly focused on selecting the devices with maximum gradient norm or loss function and requires all devices to perform local training in each round. This may produce extra training costs and schedule devices with similar data statistics, thus degrading learning performance. To mitigate these prob-lems, we proposed to schedule a subset of representative devices and find the corresponding pre-device stepsizes to approximate the full participation aggregated gradient. Taking into account the limited wireless bandwidth, we formulate an optimization problem to capture the trade-off between representativity and latency by optimizing device scheduling and bandwidth allocation policies. Our analysis reveals optimal bandwidth allocation is achieved when all scheduled devices have the same latency consisting of computation and communication latencies. Then, by proving the non-monotone sub modularity of the problem, we develop a double greedy algorithm to solve the device scheduling policy. To avoid the local training of unscheduled devices, we utilize the historical gradient information of devices to estimate the current gradient for device scheduling design. Experimental results show that the proposed latency- and representativity-aware scheduling algorithm saves over 16% and 12% training time for MNIST and CIFAR-I0 datasets than the scheduling algorithms based on either latency and representativity individually.

KW - Device scheduling

KW - resource allocation

KW - submodular optimization

KW - wireless federated Learning

UR - http://www.scopus.com/inward/record.url?scp=85146425370&partnerID=8YFLogxK

U2 - 10.1109/ICECCME55909.2022.9988590

DO - 10.1109/ICECCME55909.2022.9988590

M3 - Paper

AN - SCOPUS:85146425370

T2 - 2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2022

Y2 - 16 November 2022 through 18 November 2022

ER -